NOTE: I have been studying a lot about threading in the past few weeks and have some awesome additions in store! They will take a while to come out though. The goal of the library is still to provide a simple and efficient way to multi task in lua
Upcoming Plans: Adding network support for threading. Kinda like your own lua cloud. This will require the bin, net, and multi library. Once that happens I will include those libraries as a set. This also means that you can expect both a stand alone and joined versions of the libraries.
In Changes you’ll find documentation for(In Order):
My multitasking library for lua. It is a pure lua binding if you ingore the integrations and the love2d compat. If you find any bugs or have any issues please let me know :). If you don’t see a table of contents try using the ReadMe.html file. It is eaiser to navigate the readme
NOTE: I have been studying a lot about threading in the past few weeks and have some awesome additions in store! They will take a while to come out though. The goal of the library is still to provide a simple and efficient way to multi task in lua
Upcoming Plans: Adding network support for threading. Kinda like your own lua cloud. This will require the bin, net, and multi library. Once that happens I will include those libraries as a set. This also means that you can expect both a stand alone and joined versions of the libraries.
In Changes you’ll find documentation for(In Order):
My multitasking library for lua. It is a pure lua binding if you ingore the integrations and the love2d compat. If you find any bugs or have any issues please let me know :). If you don’t see a table of contents try using the ReadMe.html file. It is eaiser to navigate the readme
Note: The latest version of lualanes is required if you want to make use of system threads on lua 5.2+. I will update the dependencies for luarocks since this library should work fine on lua 5.2+
I still need to test though
To install copy the multi folder into your enviroment and you are good to go
or use luarocks
luarocks install multi
+INSTALLING
Note: The latest version of lualanes is required if you want to make use of system threads on lua 5.2+. I will update the dependencies for luarocks since this library should work fine on lua 5.2+
I still need to test though
To install copy the multi folder into your enviroment and you are good to go
or use luarocks
luarocks install bin -- Inorder to use the new save state stuff
+luarocks install multi
Discord
For real-time assistance with my libraries! A place where you can ask questions and get help with any of my libraries. Also you can request features and stuff there as well.
https://discord.gg/U8UspuA
Planned features/TODO
--
Add system threads for love2d that works like the lanesManager (loveManager, slight differences). -
Improve performance of the library -
Improve coroutine based threading scheduling - Add Sharable memory (Perhaps this may be impossible due to how love2d and lanes work)
- Improve love2d Idle thread cpu usage… Tricky Look at the rambling section for insight.
- Add more control to coroutine based threading
- Add more control to system based threading
- Fix the performance when using system threads in love2d
- Make practical examples that show how you can solve real problems
-
Add more features to support module creators -
Make a framework for eaiser thread task distributing -
Fix Error handling on threaded multi objects Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance! -
Add multi:OnError(function(obj,err)) - sThread.wrap(obj) May or may not be completed Theory: Allows interaction in one thread to affect it in another. The addition to threaded tables may make this possible!
- SystemThreaded Actors — After some tests i figured out a way to make this work… It will work slightly different though. This is due to the actor needing to be splittable…
- LoadBalancing for system threads (Once SystemThreaded Actors are done)
- Add more integrations
- Finish the wiki stuff. (11% done)
- Test for unknown bugs
Known Bugs/Issues
In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P
Another bug concerns the SystemThreadedJobQueue, Only 1 can be used for now… Creating more may not be a good idea.
And systemThreadedTables only supports 1 table between the main and worker thread! They do not work when shared between 2 or more threads. If you need that much flexiblity ust the GLOBAL table that all threads have.
For module creators using this library. I suggest using SystemThreadedQueues for data transfer instead of SystemThreadedTables for rapid data transfer, If you plan on having Constants that will always be the same then a table is a good idea! They support up to n threads and can be messed with and abused as much as you want :D
Love2D SystemThreadedTAbles do not send love2d userdata, use queues instead for that!
Usage:
Add system threads for love2d that works like the lanesManager (loveManager, slight differences). Improve performance of the library Improve coroutine based threading scheduling Improve love2d Idle thread cpu usage/Fix the performance when using system threads in love2d… Tricky Look at the rambling section for insight. Add more control to coroutine based threading Add more control to system based threading Make practical examples that show how you can solve real problems Add more features to support module creators Make a framework for eaiser thread task distributing Fix Error handling on threaded multi objects Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance! Add multi:OnError(function(obj,err)) sThread.wrap(obj) May or may not be completed Theory: Allows interaction in one thread to affect it in another. The addition to threaded tables may make this possible! SystemThreaded Actors — After some tests i figured out a way to make this work… It will work slightly different though. This is due to the actor needing to be splittable… LoadBalancing for system threads (Once SystemThreaded Actors are done) Add more integrations Fix SystemThreadedTables Finish the wiki stuff. (11% done) Test for unknown bugs Known Bugs/Issues
In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P
Another bug concerns the SystemThreadedJobQueue, Only 1 can be used for now… Creating more may not be a good idea.
And systemThreadedTables only supports 1 table between the main and worker thread! They do not work when shared between 2 or more threads. If you need that much flexiblity ust the GLOBAL table that all threads have.
For module creators using this library. I suggest using SystemThreadedQueues for data transfer instead of SystemThreadedTables for rapid data transfer, If you plan on having Constants that will always be the same then a table is a good idea! They support up to n threads and can be messed with and abused as much as you want :D
Love2D SystemThreadedTAbles do not send love2d userdata, use queues instead for that!
Usage:
function<
end
end)
multi:mainloop()
-
Output (Change the value inc as indicated in the comment to see the outcomes!)
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Loop timed out! tloop Trying again…
Looping…
Looping…
Looping…
Looping…
Looping…
We did it! 1 2 3
Changes
Update: 1.8.7
Added:
+
Output (Change the value inc as indicated in the comment to see the outcomes!)
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Loop timed out! tloop Trying again…
Looping…
Looping…
Looping…
Looping…
Looping…
We did it! 1 2 3
Changes
Update: 1.9.0
Added:
+- multiobj:ToString() — returns a string repersenting the object
- multi:newFromString(str) — creates an object from a string
Works on threads and regular objects. Requires the latest bin library to work!
talarm=multi:newThreadedAlarm("AlarmTest",5)
+talarm:OnRing(function()
+ print("Ring!")
+end)
+bin.new(talarm:ToString()):tofile("test.dat")
+-- multi:newFromString(bin.load("test.dat"))
+
— A more seamless way to use this will be made in the form of state saving.
This is still a WIP
processes, timers, timemasters, watchers, and queuers have not been worked on yet
Update: 1.8.7
Added:
- multi.timer(func,…)
self.sec then
@@ -379,21 +341,6 @@ function multi:benchMark(sec,p,pt)
temp.c=0
return temp
end
-function multi:tofile(path)
- local items=self:getChildren()
- io.mkDir(io.getName(path))
- for i=1,#items do
- items[i]:tofile(io.getName(path)..'\\item'..item[i]..'.dat')
- end
- local int=bin.new()
- int:addBlock('process')
- int:addBlock(io.getName(path))
- int:addBlock(#self.Mainloop)
- int:addBlock(self.Active)
- int:addBlock(self.Rest)
- int:addBlock(self.Jobs)
- int:tofile()
-end
function multi.startFPSMonitior()
if not multi.runFPS then
multi.doFPS(s)
@@ -691,6 +638,7 @@ function multi:newBase(ins)
c.funcTM={}
c.funcTMR={}
c.ender={}
+ c.important={}
c.Id=0
c.PId=0
c.Act=function() end
@@ -801,7 +749,7 @@ function multi:newTimer()
return (os.clock()-self.time)+self.count
end
function c:isPaused()
- return c.paused
+ return self.paused
end
c.Reset=c.Start
function c:Pause()
@@ -1158,14 +1106,6 @@ function multi:newEvent(task)
function c:OnEvent(func)
table.insert(self.func,func)
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.Task)
- m:addBlock(self.func)
- m:addBlock(self.Active)
- m:tofile(path)
- end
self:create(c)
return c
end
@@ -1196,13 +1136,6 @@ function multi:newAlarm(set)
c.Priority=self.Priority_Low
c.timer=self:newTimer()
c.set=set or 0
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.set)
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Act()
if self.timer:Get()>=self.set then
self:Pause()
@@ -1238,13 +1171,6 @@ function multi:newLoop(func)
if func then
c.func={func}
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Act()
for i=1,#self.func do
self.func[i](self,self.Parent.clock()-self.Start)
@@ -1291,16 +1217,6 @@ function multi:newStep(start,reset,count,skip)
think=-1
end
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.funcE)
- m:addBlock(self.funcS)
- m:addBlock({pos=self.pos,endAt=self.endAt,skip=self.skip,spos=self.spos,count=self.count,start=self.start})
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Act()
if self~=nil then
if self.spos==0 then
@@ -1362,13 +1278,6 @@ function multi:newTLoop(func,set)
if func then
c.func={func}
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Act()
if self.timer:Get()>=self.set then
self.life=self.life+1
@@ -1399,12 +1308,6 @@ function multi:newTrigger(func)
function c:Fire(...)
self:trigfunc(...)
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.trigfunc)
- m:tofile(path)
- end
self:create(c)
return c
end
@@ -1432,16 +1335,6 @@ function multi:newTStep(start,reset,count,set)
self.timer=self.clock()
self:Resume()
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.funcE)
- m:addBlock(self.funcS)
- m:addBlock({pos=self.pos,endAt=self.endAt,skip=self.skip,timer=self.timer,count=self.count,start=self.start,set=self.set})
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Act()
if self.clock()-self.timer>=self.set then
self:Reset()
@@ -1557,15 +1450,18 @@ function thread.testFor(name,val,sym)
thread.hold(function() return thread.get(name)~=nil end)
return thread.get(name)
end
-function multi:newTBase(ins)
+function multi:newTBase(name)
local c = {}
+ c.name=name
c.Active=true
c.func={}
c.ender={}
c.Id=0
c.PId=0
c.Parent=self
+ c.important={}
c.held=false
+ c.ToString=multi.ToString
return c
end
function multi:newThread(name,func)
@@ -1679,17 +1575,10 @@ end)
multi.scheduler:Pause()
multi.OnError=multi:newConnection()
function multi:newThreadedAlarm(name,set)
- local c=self:newTBase()
+ local c=self:newTBase(name)
c.Type='alarmThread'
c.timer=self:newTimer()
c.set=set or 0
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.set)
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Resume()
self.rest=false
self.timer:Resume()
@@ -1728,7 +1617,7 @@ function multi:newThreadedAlarm(name,set)
return c
end
function multi:newThreadedUpdater(name,skip)
- local c=self:newTBase()
+ local c=self:newTBase(name)
c.Type='updaterThread'
c.pos=1
c.skip=skip or 1
@@ -1759,10 +1648,9 @@ function multi:newThreadedUpdater(name,skip)
return c
end
function multi:newThreadedTStep(name,start,reset,count,set)
- local c=self:newTBase()
+ local c=self:newTBase(name)
local think=1
c.Type='tstepThread'
- c.Priority=self.Priority_Low
c.start=start or 1
local reset = reset or math.huge
c.endAt=reset
@@ -1782,16 +1670,6 @@ function multi:newThreadedTStep(name,start,reset,count,set)
self.timer=os.clock()
self:Resume()
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.funcE)
- m:addBlock(self.funcS)
- m:addBlock({pos=self.pos,endAt=self.endAt,skip=self.skip,timer=self.timer,count=self.count,start=self.start,set=self.set})
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Resume()
self.rest=false
end
@@ -1830,7 +1708,7 @@ function multi:newThreadedTStep(name,start,reset,count,set)
end
end
for i=1,#c.func do
- c.func[i](c.pos,c)
+ c.func[i](c,c.pos)
end
c.pos=c.pos+c.count
if c.pos-c.count==c.endAt then
@@ -1849,19 +1727,12 @@ function multi:newThreadedTStep(name,start,reset,count,set)
return c
end
function multi:newThreadedTLoop(name,func,n)
- local c=self:newTBase()
+ local c=self:newTBase(name)
c.Type='tloopThread'
c.restN=n or 1
if func then
c.func={func}
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Resume()
self.rest=false
end
@@ -1890,7 +1761,7 @@ function multi:newThreadedTLoop(name,func,n)
return c
end
function multi:newThreadedStep(name,start,reset,count,skip)
- local c=self:newTBase()
+ local c=self:newTBase(name)
local think=1
c.Type='stepThread'
c.pos=start or 1
@@ -1906,16 +1777,6 @@ function multi:newThreadedStep(name,start,reset,count,skip)
think=-1
end
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.funcE)
- m:addBlock(self.funcS)
- m:addBlock({pos=self.pos,endAt=self.endAt,skip=self.skip,spos=self.spos,count=self.count,start=self.start})
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Resume()
self.rest=false
end
@@ -1957,7 +1818,7 @@ function multi:newThreadedStep(name,start,reset,count,skip)
end
end
for i=1,#c.func do
- c.func[i](c.pos,c)
+ c.func[i](c,c.pos)
end
c.pos=c.pos+c.count
if c.pos-c.count==c.endAt then
@@ -2063,19 +1924,12 @@ function multi:newThreadedProcess(name)
return c
end
function multi:newThreadedLoop(name,func)
- local c=self:newTBase()
+ local c=self:newTBase(name)
c.Type='loopThread'
c.Start=os.clock()
if func then
c.func={func}
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.func)
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Resume()
self.rest=false
end
@@ -2104,20 +1958,12 @@ function multi:newThreadedLoop(name,func)
return c
end
function multi:newThreadedEvent(name,task)
- local c=self:newTBase()
+ local c=self:newTBase(name)
c.Type='eventThread'
c.Task=task or function() end
function c:OnEvent(func)
table.insert(self.func,func)
end
- function c:tofile(path)
- local m=bin.new()
- m:addBlock(self.Type)
- m:addBlock(self.Task)
- m:addBlock(self.func)
- m:addBlock(self.Active)
- m:tofile(path)
- end
function c:Resume()
self.rest=false
end
@@ -2145,3 +1991,203 @@ function multi:newThreadedEvent(name,task)
self:create(c)
return c
end
+-- State Saving Stuff
+function multi:ToString()
+ local t=self.Type
+ local data;
+ if t:sub(-6)=="Thread" then
+ data={
+ Type=t,
+ rest=self.rest,
+ updaterate=self.updaterest,
+ restrate=self.restrate,
+ name=self.name,
+ func=self.func,
+ important=self.important,
+ Active=self.Active,
+ ender=self.ender,
+ -- IDK if these need to be present...
+ -- Id=self.Id,
+ -- PId=self.PId,
+ held=self.held,
+ }
+ else
+ data={
+ Type=t,
+ func=self.func,
+ funcTM=self.funcTM,
+ funcTMR=self.funcTMR,
+ important=self.important,
+ ender=self.ender,
+ -- IDK if these need to be present...
+ -- Id=self.Id,
+ -- PId=self.PId,
+ held=self.held,
+ }
+ end
+ if t=="process" then
+ --
+ elseif t=="eventThread" or t=="event" then
+ table.merge(data,{
+ Task=self.Task,
+ })
+ elseif t=="loopThread" or t=="loop" then
+ table.merge(data,{
+ Start=self.Start,
+ })
+ elseif t=="stepThread" or t=="step" then
+ table.merge(data,{
+ funcE=self.funcE,
+ funcS=self.funcS,
+ pos=self.pos,
+ endAt=self.endAt,
+ start=self.start,
+ spos=self.spos,
+ skip=self.skip,
+ count=self.count,
+ })
+ elseif t=="tloopThread" then
+ table.merge(data,{
+ restN=self.restN,
+ })
+ elseif t=="tloop" then
+ table.merge(data,{
+ set=self.set,
+ life=self.life,
+ })
+ elseif t=="tstepThread" or t=="tstep" then
+ table.merge(data,{
+ funcE=self.funcE,
+ funcS=self.funcS,
+ pos=self.pos,
+ endAt=self.endAt,
+ start=self.start,
+ spos=self.spos,
+ skip=self.skip,
+ count=self.count,
+ timer=self.timer,
+ set=self.set,
+ reset=self.reset,
+ })
+ elseif t=="updaterThread" or t=="updater" then
+ table.merge(data,{
+ pos=self.pos,
+ skip=self.skip,
+ })
+ elseif t=="alarmThread" or t=="alarm" then
+ table.merge(data,{
+ set=self.set,
+ })
+ elseif t=="watcher" then
+ print("Currently cannot sterilize a watcher object!")
+ -- needs testing
+ -- table.merge(data,{
+ -- ns=self.ns,
+ -- n=self.n,
+ -- cv=self.cv,
+ -- })
+ elseif t=="timemaster" then
+ -- Weird stuff is going on here!
+ -- Need to do some testing
+ table.merge(data,{
+ timer=self.timer,
+ _timer=self._timer,
+ set=self.set,
+ link=self.link,
+ })
+ end
+ for i,v in pairs(self.important) do
+ data[v]=self[v]
+ end
+ local str=bin.new()
+ str:addBlock(data)
+ return str.data
+end
+function multi:newFromString(str)
+ if type(str)=="table" then
+ if str.Type=="bin" then
+ str=str.data
+ end
+ end
+ local data=bin.new(str):getBlock("t")
+ local t=data.Type
+ if t=="step" then -- GOOD
+ local item=self:newStep()
+ table.merge(item,data)
+ return item
+ elseif t=="tstep" then -- GOOD
+ local item=self:newTStep()
+ table.merge(item,data)
+ return item
+ elseif t=="tloop" then -- GOOD
+ local item=self:newTLoop()
+ table.merge(item,data)
+ return item
+ elseif t=="event" then -- GOOD
+ local item=self:newEvent(data.task)
+ table.merge(item,data)
+ return item
+ elseif t=="alarm" then -- GOOD
+ local item=self:newAlarm()
+ table.merge(item,data)
+ return item
+ elseif t=="watcher" then -- NEEDS TESTING
+ local item=self:newWatcher()
+ table.merge(item,data)
+ return item
+ elseif t=="updater" then -- GOOD
+ local item=self:newUpdater()
+ table.merge(item,data)
+ return item
+ elseif t=="loop" then -- GOOD
+ local item=self:newLoop()
+ table.merge(item,data)
+ return item
+ elseif t=="eventThread" then -- GOOD
+ local item=self:newThreadedEvent(data.name)
+ table.merge(item,data)
+ return item
+ elseif t=="loopThread" then -- GOOD
+ local item=self:newThreadedLoop(data.name)
+ table.merge(item,data)
+ return item
+ elseif t=="stepThread" then -- GOOD
+ local item=self:newThreadedStep(data.name)
+ table.merge(item,data)
+ return item
+ elseif t=="tloopThread" then -- GOOD
+ local item=self:newThreadedTLoop(data.name,nil,data.restN)
+ table.merge(item,data)
+ return item
+ elseif t=="tstepThread" then -- GOOD
+ local item=self:newThreadedTStep(data.name)
+ table.merge(item,data)
+ return item
+ elseif t=="updaterThread" then -- GOOD
+ local item=self:newThreadedUpdater(data.name)
+ table.merge(item,data)
+ return item
+ elseif t=="alarmThread" then -- GOOD
+ local item=self:newThreadedAlarm(data.name)
+ table.merge(item,data)
+ return item
+ end
+end
+function multi:Important(varname)
+ table.insert(important,varname)
+end
+function multi:SetStateFlag(opt)
+ --
+end
+function multi:quickStateSave(b)
+ --
+end
+function multi:saveState(path,opt)
+ --
+end
+function multi:loadState(path)
+ --
+end
+function multi:setDefualtStateFlag(opt)
+ --
+end
\ No newline at end of file
diff --git a/multi/integration/lanesManager.lua b/multi/integration/lanesManager.lua
index a490ce8..ee1126a 100644
--- a/multi/integration/lanesManager.lua
+++ b/multi/integration/lanesManager.lua
@@ -21,6 +21,7 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
]]
+package.path="?/init.lua;?.lua;"..package.path
function os.getOS()
if package.config:sub(1,1)=='\\' then
return 'windows'
diff --git a/test.dat b/test.dat
new file mode 100644
index 0000000..03e1c8e
Binary files /dev/null and b/test.dat differ