From 03c91d84e329896fde79c08edd630bce5856c548 Mon Sep 17 00:00:00 2001 From: Ryan Date: Sun, 2 Jul 2017 20:00:17 -0400 Subject: [PATCH 1/5] Doing tests... Love2d Still unstable with new update --- README.html | 6 +- README.md | 8 +-- multi/init.lua | 13 +++-- multi/integration/lanesManager.lua | 11 ++-- multi/integration/loveManager.lua | 46 ++++++++------- multi/integration/shared/shared.lua | 89 ++++++++++++++++++++++++++--- rockspecs/multi-1.8-2.rockspec | 2 +- rockspecs/multi-1.8-3.rockspec | 2 +- 8 files changed, 126 insertions(+), 51 deletions(-) diff --git a/README.html b/README.html index b5fd3f5..4dd3bb4 100644 --- a/README.html +++ b/README.html @@ -186,7 +186,7 @@

INSTALLING

luarocks install multi
 

Discord

For real-time assistance with my libraries! A place where you can ask questions and get help with any of my libraries. Also you can request features and stuff there as well.

https://discord.gg/U8UspuA

Planned features/TODO

Known Bugs/Issues

In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P

Usage:

 Add system threads for love2d that works like the lanesManager (loveManager, slight differences).
  • Improve performance of the library
  • Improve coroutine based threading scheduling
  • Add more features to support module creators
  • Make a framework for eaiser thread task distributing
  • Fix Error handling on threaded multi objects Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance!
  • Add multi:OnError(function(obj,err))
  • sThread.wrap(obj) May or may not be completed Theory: Allows interaction in one thread to affect it in another. The addition to threaded tables may make this possible!
  • SystemThreaded Actors — After some tests i figured out a way to make this work… It will work slightly different though. This is due to the actor needing to be splittable…
  • LoadBalancing for system threads (Once SystemThreaded Actors are done)
  • Add more integrations
  • Finish the wiki stuff. (11% done)
  • Test for unknown bugs
  • Known Bugs/Issues

    In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P

    Usage:

    functionend)
     multi:mainloop() -- the main loop of the program, multi:umanager() exists as well to allow integration in other loops Ex: love2d love.update function. More on this binding in the wiki!
     

    The library is modular so you only need to require what you need to. Because of this, the global enviroment is altered

    There are many useful objects that you can use

    Check out the wiki for detailed usage, but here are the objects:

      -
    • Process#
    • Queuer#
    • Alarm
    • Loop
    • Event
    • Step
    • Range
    • TStep
    • TLoop
    • Condition
    • Connection
    • Timer
    • Updater
    • Thread*
    • Trigger
    • Task
    • Job
    • Function
    • Watcher

      Note: Both a process and queue act like the multi namespace, but allows for some cool things. Because they use the other objects an example on them will be done last

      *Uses the built in coroutine features of lua, these have an interesting interaction with the other means of multi-tasking

      Triggers are kind of useless after the creation of the Connection

      Watchers have no real purpose as well I made it just because.

    Examples of each object being used

    We already showed alarms in action so lets move on to a Loop object

    Throughout these examples I am going to do some strange things in order to show other features of the library!

    LOOPS

    Examples of each object being used

    We already showed alarms in action so lets move on to a Loop object

    Throughout these examples I am going to do some strange things in order to show other features of the library!

    LOOPS

    3):OnBench(3,"All Threads: ")
     end)
     multi:mainloop()
    -

    Using multi:newSystemThreadedQueue()

    Quick Note: queues shared across multiple objects will be pulling from the same “queue” keep this in mind when coding! Also the queue respects direction a push on the thread side cannot be popped on the thread side… Same goes for the mainthread!

    Using multi:newSystemThreadedQueue()

    Quick Note: queues shared across multiple objects will be pulling from the same “queue” keep this in mind when coding! Also the queue respects direction a push on the thread side cannot be popped on the thread side… Same goes for the mainthread!
    Turns out i was wrong about this…

    
    +Quick Note: queues shared across multiple objects will be pulling from the same "queue" keep this in mind when coding! ~~Also the queue respects direction a push on the thread side cannot be popped on the thread side... Same goes for the mainthread!
    ~~ Turns out i was wrong about this... ```lua -- in love2d, this file will be in the same example folder as before, but is named main2.lua require("core.Library") diff --git a/multi/init.lua b/multi/init.lua index 3bd3e8d..42f8b74 100644 --- a/multi/init.lua +++ b/multi/init.lua @@ -45,8 +45,8 @@ function print(...) end end multi = {} -multi.Version="1.8.3" -multi._VERSION="1.8.3" +multi.Version="1.8.4" +multi._VERSION="1.8.4" multi.stage='stable' multi.__index = multi multi.Mainloop={} @@ -980,6 +980,7 @@ function multi:mainloop() end end function multi:protectedMainloop() + multi:protect() if not multi.isRunning then multi.isRunning=true for i=1,#self.Tasks do @@ -994,6 +995,7 @@ function multi:protectedMainloop() end end function multi:unprotectedMainloop() + multi:unProtect() if not multi.isRunning then multi.isRunning=true for i=1,#self.Tasks do @@ -1018,6 +1020,7 @@ function multi:unprotectedMainloop() end end function multi:prioritizedMainloop1() + multi:enablePriority() if not multi.isRunning then multi.isRunning=true for i=1,#self.Tasks do @@ -1049,6 +1052,7 @@ function multi:prioritizedMainloop1() end end function multi:prioritizedMainloop2() + multi:enablePriority2() if not multi.isRunning then multi.isRunning=true for i=1,#self.Tasks do @@ -1593,7 +1597,7 @@ function multi:newThread(name,func) end multi:setDomainName("Threads") multi:setDomainName("Globals") -multi.scheduler=multi:newUpdater() +multi.scheduler=multi:newLoop() multi.scheduler.Type="scheduler" function multi.scheduler:setStep(n) self.skip=tonumber(n) or 24 @@ -1602,7 +1606,7 @@ multi.scheduler.skip=0 multi.scheduler.counter=0 multi.scheduler.Threads=multi:linkDomain("Threads") multi.scheduler.Globals=multi:linkDomain("Globals") -multi.scheduler:OnUpdate(function(self) +multi.scheduler:OnLoop(function(self) self.counter=self.counter+1 for i=#self.Threads,1,-1 do ret={} @@ -1656,7 +1660,6 @@ multi.scheduler:OnUpdate(function(self) end end end) -multi.scheduler:setStep() multi.scheduler:Pause() multi.OnError=multi:newConnection() function multi:newThreadedAlarm(name,set) diff --git a/multi/integration/lanesManager.lua b/multi/integration/lanesManager.lua index 49a81f8..9c0e60b 100644 --- a/multi/integration/lanesManager.lua +++ b/multi/integration/lanesManager.lua @@ -125,12 +125,9 @@ function multi:newSystemThread(name,func) end) return c end -print("Intergrated Lanes!") -multi.intergration={} -- for module creators -multi.intergration.GLOBAL=GLOBAL -multi.intergration.THREAD=THREAD -multi.intergration.lanes={} -multi.intergration.lanes.GLOBAL=GLOBAL -- for module creators -multi.intergration.lanes.THREAD=THREAD -- for module creators +print("Integrated Lanes!") +multi.integration={} -- for module creators +multi.integration.GLOBAL=GLOBAL +multi.integration.THREAD=THREAD require("multi.integration.shared.shared") return {init=function() return GLOBAL,THREAD end} diff --git a/multi/integration/loveManager.lua b/multi/integration/loveManager.lua index 7422f38..e357732 100644 --- a/multi/integration/loveManager.lua +++ b/multi/integration/loveManager.lua @@ -2,14 +2,14 @@ require("multi.compat.love2d") function multi:canSystemThread() return true end -multi.intergration={} -multi.intergration.love2d={} -multi.intergration.love2d.ThreadBase=[[ +multi.integration={} +multi.integration.love2d={} +multi.integration.love2d.ThreadBase=[[ __THREADNAME__=({...})[1] require("love.filesystem") require("love.system") require("love.timer") -require("multi.all") +require("multi") GLOBAL={} setmetatable(GLOBAL,{ __index=function(t,k) @@ -79,7 +79,7 @@ function resolveType(tp,d) elseif tp=="bool" then return (d=="true") elseif tp=="function" then - return loadDump(d) + return loadDump("[==["..d.."]==]") elseif tp=="table" then return loadstring("return "..d)() elseif tp=="nil" then @@ -91,11 +91,11 @@ end function resolveData(v) local data="" if type(v)=="table" then - data=ToStr(v) + return ToStr(v) elseif type(v)=="function" then - data=dump(v) + return dump(v) elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then - data=tostring(v) + return tostring(v) end return data end @@ -191,7 +191,7 @@ setmetatable(GLOBAL,{ end, }) THREAD={} -- Allow main thread to interact with these objects as well -multi.intergration.love2d.mainChannel=love.thread.getChannel("__MainChan__") +multi.integration.love2d.mainChannel=love.thread.getChannel("__MainChan__") function ToStr(val, name, skipnewlines, depth) skipnewlines = skipnewlines or false depth = depth or 0 @@ -228,7 +228,7 @@ function resolveType(tp,d) elseif tp=="bool" then return (d=="true") elseif tp=="function" then - return loadDump(d) + return loadDump("[==["..d.."]==]") elseif tp=="table" then return loadstring("return "..d)() elseif tp=="nil" then @@ -240,11 +240,11 @@ end function resolveData(v) local data="" if type(v)=="table" then - data=ToStr(v) + return ToStr(v) elseif type(v)=="function" then - data=dump(v) + return dump(v) elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then - data=tostring(v) + return tostring(v) end return data end @@ -274,10 +274,10 @@ function multi:newSystemThread(name,func) -- the main method local c={} c.name=name c.ID=c.name.."" - c.thread=love.thread.newThread(multi.intergration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func))) + c.thread=love.thread.newThread(multi.integration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func))) c.thread:start(c.ID) function c:kill() - multi.intergration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__" + multi.integration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__" end return c end @@ -310,11 +310,11 @@ function THREAD.hold(n) multi.OBJ_REF:Resume() end __channels__={} -multi.intergration.GLOBAL=GLOBAL -multi.intergration.THREAD=THREAD +multi.integration.GLOBAL=GLOBAL +multi.integration.THREAD=THREAD updater=multi:newUpdater() updater:OnUpdate(function(self) - local data=multi.intergration.love2d.mainChannel:pop() + local data=multi.integration.love2d.mainChannel:pop() while data do --print("MAIN:",data) if type(data)=="string" then @@ -342,19 +342,21 @@ updater:OnUpdate(function(self) else __proxy__[name]=data end - data=multi.intergration.love2d.mainChannel:pop() + data=multi.integration.love2d.mainChannel:pop() end end) require("multi.integration.shared.shared") -print("Intergrated Love2d!") +print("Integrated Love2d!") return { init=function(t) if t then if t.threadNamespace then - multi.intergration.love2d.ThreadBase:gsub("sThread",t.threadNamespace) + multi.integration.THREADNAME=t.threadNamespace + multi.integration.love2d.ThreadBase:gsub("sThread",t.threadNamespace) end if t.globalNamespace then - multi.intergration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace) + multi.integration.GLOBALNAME=t.globalNamespace + multi.integration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace) end end return GLOBAL,THREAD diff --git a/multi/integration/shared/shared.lua b/multi/integration/shared/shared.lua index 023b38f..b393612 100644 --- a/multi/integration/shared/shared.lua +++ b/multi/integration/shared/shared.lua @@ -29,10 +29,16 @@ function multi:newSystemThreadedQueue(name) -- in love2d this will spawn a chann function c:init() -- create an init function so we can mimic on bith love2d and lanes self.chan=love.thread.getChannel(self.name) -- create channel by the name self.name function self:push(v) -- push to the channel - self.chan:push(v) + self.chan:push({type(v),resolveData(v)}) end function self:pop() -- pop from the channel - return self.chan:pop() + local tab=self.chan:pop() + if not tab then return end + return resolveType(tab[1],tab[2]) + end + function self:peek() + local tp,d=unpack{self.chan:peek()} + return resolveType(tp,d) end GLOBAL[self.name]=self -- send the object to the thread through the global interface return self -- return the object @@ -49,20 +55,23 @@ function multi:newSystemThreadedQueue(name) -- in love2d this will spawn a chann function c:pop() -- pop the queue return ({self.linda:receive(0,"Q")})[2] end + function c:peek() + return self.linda:get("Q") + end function c:init() -- mimic the feature that love2d requires, so code can be consistent return self end - multi.intergration.GLOBAL[name]=c -- send the object to the thread through the global interface + multi.integration.GLOBAL[name]=c -- send the object to the thread through the global interface end return c end function multi:systemThreadedBenchmark(n,p) n=n or 1 - local cores=multi.intergration.THREAD.getCores() + local cores=multi.integration.THREAD.getCores() local queue=multi:newSystemThreadedQueue("QUEUE") - multi.intergration.GLOBAL["__SYSTEMBENCHMARK__"]=n - local sThread=multi.intergration.THREAD - local GLOBAL=multi.intergration.GLOBAL + multi.integration.GLOBAL["__SYSTEMBENCHMARK__"]=n + local sThread=multi.integration.THREAD + local GLOBAL=multi.integration.GLOBAL for i=1,cores do multi:newSystemThread("STHREAD_BENCH",function() require("multi") @@ -170,7 +179,71 @@ function multi:newSystemThreadedTable(name) }) return self end - multi.intergration.GLOBAL[name]=c -- send the object to the thread through the global interface + multi.integration.GLOBAL[name]=c -- send the object to the thread through the global interface end return c end +function multi:newSystemThreadedJobQueue(numOfCores) + local c={} + c.jobnum=1 + c.cores=numOfCores or multi.integration.THREAD.getCores() + c.queueIN=multi:newSystemThreadedQueue("THREADED_JQ"):init() + c.queueOUT=multi:newSystemThreadedQueue("THREADED_JQO"):init() + c.REG=multi:newSystemThreadedTable("THREADED_JQ_F_REG"):init() + -- registerJob(name,func) + -- pushJob(...) + function c:registerJob(name,func) + self.REG[name]=func + end + function c:pushJob(name,...) + self.queueOUT:push({self.jobnum,name,...}) + self.jobnum=self.jobnum+1 + end + local GLOBAL=multi.integration.GLOBAL -- set up locals incase we are using lanes + local sThread=multi.integration.THREAD -- set up locals incase we are using lanes + GLOBAL["__JQ_COUNT__"]=c.cores + for i=1,c.cores do + multi:newSystemThread("System Threaded Job Queue Worker Thread #"..i,function() + require("multi") + __sleep__=.001 + if love then -- lets make sure we don't reference upvalues if using love2d + GLOBAL=_G.GLOBAL + sThread=_G.sThread + __sleep__=.1 + end + JQI=sThread.waitFor("THREADED_JQO"):init() -- Grab it + JQO=sThread.waitFor("THREADED_JQ"):init() -- Grab it + FGLOBAL=sThread.waitFor("THREADED_JQ_F_REG"):init() -- Grab it + sThread.sleep(.1) -- lets wait for things to work out + setmetatable(_G,{ + __index=FGLOBAL + }) + GLOBAL["THREADED_JQ"]=nil -- remove it + GLOBAL["THREADED_JQO"]=nil -- remove it + GLOBAL["THREADED_JQ_F_REG"]=nil -- remove it + multi:newLoop(function() + sThread.sleep(__sleep__) -- lets allow cpu time for other processes on our system! + local job=JQI:pop() + if job then + local ID=table.remove(job,1) -- return and remove + local name=table.remove(job,1) -- return and remove + local ret={FGLOBAL:waitFor(name)(unpack(job))} -- unpack the rest + JQO:push({ID,ret}) + end + end) + multi:mainloop() + end) + end + c.OnJobCompleted=multi:newConnection() + c.updater=multi:newLoop(function(self) + local data=self.link.queueIN:pop() + while data do + if data then + self.link.OnJobCompleted:Fire(unpack(data)) + end + data=self.link.queueIN:pop() + end + end) + c.updater.link=c + return c +end diff --git a/rockspecs/multi-1.8-2.rockspec b/rockspecs/multi-1.8-2.rockspec index 2125f12..6ffbdf9 100644 --- a/rockspecs/multi-1.8-2.rockspec +++ b/rockspecs/multi-1.8-2.rockspec @@ -22,7 +22,7 @@ build = { ["multi.init"] = "multi/init.lua", ["multi.all"] = "multi/all.lua", ["multi.compat.backwards[1,5,0]"] = "multi/compat/backwards[1,5,0].lua", - ["multi.compat"] = "multi/compat/love2d.lua", + ["multi.compat.love2d"] = "multi/compat/love2d.lua", ["multi.integration.lanesManager"] = "multi/integration/lanesManager.lua", ["multi.integration.loveManager"] = "multi/integration/loveManager.lua", ["multi.integration.shared.shared"] = "multi/integration/shared/shared.lua" diff --git a/rockspecs/multi-1.8-3.rockspec b/rockspecs/multi-1.8-3.rockspec index 88088ad..e3e152f 100644 --- a/rockspecs/multi-1.8-3.rockspec +++ b/rockspecs/multi-1.8-3.rockspec @@ -22,7 +22,7 @@ build = { ["multi.init"] = "multi/init.lua", ["multi.all"] = "multi/all.lua", ["multi.compat.backwards[1,5,0]"] = "multi/compat/backwards[1,5,0].lua", - ["multi.compat"] = "multi/compat/love2d.lua", + ["multi.compat.love2d"] = "multi/compat/love2d.lua", ["multi.integration.lanesManager"] = "multi/integration/lanesManager.lua", ["multi.integration.loveManager"] = "multi/integration/loveManager.lua", ["multi.integration.shared.shared"] = "multi/integration/shared/shared.lua" -- 2.43.0 From fe123f879ed84822c2270a0998d298259f23c1b8 Mon Sep 17 00:00:00 2001 From: Ryan Date: Sun, 2 Jul 2017 22:55:46 -0400 Subject: [PATCH 2/5] Still fixing bugs --- ...ergratetest.lua => lanesintegratetest.lua} | 0 ...gratetest2.lua => lanesintegratetest2.lua} | 0 ...gratetest3.lua => lanesintegratetest3.lua} | 0 ...gratetest4.lua => lanesintegratetest4.lua} | 0 ...gratetest5.lua => lanesintegratetest5.lua} | 0 ...gratetest6.lua => lanesintegratetest6.lua} | 0 examples/lanesintegratetest7.lua | 26 +++ examples/love2d Threading Example/main5.lua | 52 ++++++ .../multi/integration/shared/shared.lua | 148 +++++++++++++++++- multi/integration/loveManager.lua | 19 +-- 10 files changed, 220 insertions(+), 25 deletions(-) rename examples/{lanesintergratetest.lua => lanesintegratetest.lua} (100%) rename examples/{lanesintergratetest2.lua => lanesintegratetest2.lua} (100%) rename examples/{lanesintergratetest3.lua => lanesintegratetest3.lua} (100%) rename examples/{lanesintergratetest4.lua => lanesintegratetest4.lua} (100%) rename examples/{lanesintergratetest5.lua => lanesintegratetest5.lua} (100%) rename examples/{lanesintergratetest6.lua => lanesintegratetest6.lua} (100%) create mode 100644 examples/lanesintegratetest7.lua create mode 100644 examples/love2d Threading Example/main5.lua diff --git a/examples/lanesintergratetest.lua b/examples/lanesintegratetest.lua similarity index 100% rename from examples/lanesintergratetest.lua rename to examples/lanesintegratetest.lua diff --git a/examples/lanesintergratetest2.lua b/examples/lanesintegratetest2.lua similarity index 100% rename from examples/lanesintergratetest2.lua rename to examples/lanesintegratetest2.lua diff --git a/examples/lanesintergratetest3.lua b/examples/lanesintegratetest3.lua similarity index 100% rename from examples/lanesintergratetest3.lua rename to examples/lanesintegratetest3.lua diff --git a/examples/lanesintergratetest4.lua b/examples/lanesintegratetest4.lua similarity index 100% rename from examples/lanesintergratetest4.lua rename to examples/lanesintegratetest4.lua diff --git a/examples/lanesintergratetest5.lua b/examples/lanesintegratetest5.lua similarity index 100% rename from examples/lanesintergratetest5.lua rename to examples/lanesintegratetest5.lua diff --git a/examples/lanesintergratetest6.lua b/examples/lanesintegratetest6.lua similarity index 100% rename from examples/lanesintergratetest6.lua rename to examples/lanesintegratetest6.lua diff --git a/examples/lanesintegratetest7.lua b/examples/lanesintegratetest7.lua new file mode 100644 index 0000000..7a9f2df --- /dev/null +++ b/examples/lanesintegratetest7.lua @@ -0,0 +1,26 @@ +package.path="?/init.lua;"..package.path +local GLOBAL,sThread=require("multi.integration.lanesManager").init() +jQueue=multi:newSystemThreadedJobQueue() -- this internally creates System threads, We told it to use a maximum of 3 cores at any given time +jQueue:registerJob("TEST_JOB",function(a,s) + math.randomseed(s) + -- We will push a random # + TEST_JOB2() -- You can call other registered functions as well! + return math.random(0,255) -- send the result to the main thread +end) +jQueue:registerJob("TEST_JOB2",function() + print("Test Works!") +end) +tableOfOrder={} +jQueue.OnJobCompleted(function(JOBID,n) + -- JOBID is the completed job, starts at 1 and counts up by 1. + -- Threads finish at different times so jobids may be returned out of order! Be sure to have a way to order them + tableOfOrder[JOBID]=n -- we order ours by putting them into a table + if #tableOfOrder==10 then + print("We got all of the pieces!") + end +end) +for i=1,10 do -- Job Name of registered function, ... varargs + jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000)) +end +print("I pushed all of the jobs :)") +multi:mainloop() diff --git a/examples/love2d Threading Example/main5.lua b/examples/love2d Threading Example/main5.lua new file mode 100644 index 0000000..3c50850 --- /dev/null +++ b/examples/love2d Threading Example/main5.lua @@ -0,0 +1,52 @@ +--~ require("core.Library") +--~ GLOBAL,sThread=require("multi.integration.loveManager").init() -- load the love2d version of the lanesManager and requires the entire multi library +--~ require("core.GuiManager") +--~ gui.ff.Color=Color.Black +--~ test=multi:newSystemThreadedTable("YO"):init() +--~ test["test1"]="lol" +--~ multi:newSystemThread("test",function() +--~ tab=sThread.waitFor("YO"):init() +--~ print(tab["test1"]) +--~ sThread.sleep(3) +--~ tab["test2"]="Whats so funny?" +--~ end) +--~ multi:newThread("test2",function() +--~ thread.sleep(1) +--~ print(test:waitFor("test2")) +--~ t.text="DONE!" +--~ end) +--~ t=gui:newTextLabel("no done yet!",0,0,300,100) +--~ t:centerX() +--~ t:centerY() + +require("core.Library") +GLOBAL,sThread=require("multi.integration.loveManager").init() -- load the love2d version of the lanesManager and requires the entire multi library +require("core.GuiManager") +gui.ff.Color=Color.Black +jQueue=multi:newSystemThreadedJobQueue() -- this internally creates System threads, We told it to use a maximum of 3 cores at any given time +jQueue:registerJob("TEST_JOB",function(a,s) + math.randomseed(s) + print("testing...") + -- We will push a random # + TEST_JOB2() -- You can call other registered functions as well! + return math.random(0,255) -- send the result to the main thread +end) +jQueue:registerJob("TEST_JOB2",function(a,s) + print("Test Works!") +end) +tableOfOrder={} +jQueue.OnJobCompleted(function(JOBID,n) + -- JOBID is the completed job, starts at 1 and counts up by 1. + -- Threads finish at different times so jobids may be returned out of order! Be sure to have a way to order them + tableOfOrder[JOBID]=n -- we order ours by putting them into a table + if #tableOfOrder==10 then + print("We got all of the pieces!") + end +end) +for i=1,10 do -- Job Name of registered function, ... varargs + jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000)) +end +print("I pushed all of the jobs :)") +t=gui:newTextLabel("no done yet!",0,0,300,100) +t:centerX() +t:centerY() diff --git a/examples/love2d Threading Example/multi/integration/shared/shared.lua b/examples/love2d Threading Example/multi/integration/shared/shared.lua index 023b38f..06bb8cf 100644 --- a/examples/love2d Threading Example/multi/integration/shared/shared.lua +++ b/examples/love2d Threading Example/multi/integration/shared/shared.lua @@ -29,10 +29,16 @@ function multi:newSystemThreadedQueue(name) -- in love2d this will spawn a chann function c:init() -- create an init function so we can mimic on bith love2d and lanes self.chan=love.thread.getChannel(self.name) -- create channel by the name self.name function self:push(v) -- push to the channel - self.chan:push(v) + self.chan:push({type(v),resolveData(v)}) end function self:pop() -- pop from the channel - return self.chan:pop() + local tab=self.chan:pop() + if not tab then return end + return resolveType(tab[1],tab[2]) + end + function self:peek() + local tp,d=unpack{self.chan:peek()} + return resolveType(tp,d) end GLOBAL[self.name]=self -- send the object to the thread through the global interface return self -- return the object @@ -49,20 +55,23 @@ function multi:newSystemThreadedQueue(name) -- in love2d this will spawn a chann function c:pop() -- pop the queue return ({self.linda:receive(0,"Q")})[2] end + function c:peek() + return self.linda:get("Q") + end function c:init() -- mimic the feature that love2d requires, so code can be consistent return self end - multi.intergration.GLOBAL[name]=c -- send the object to the thread through the global interface + multi.integration.GLOBAL[name]=c -- send the object to the thread through the global interface end return c end function multi:systemThreadedBenchmark(n,p) n=n or 1 - local cores=multi.intergration.THREAD.getCores() + local cores=multi.integration.THREAD.getCores() local queue=multi:newSystemThreadedQueue("QUEUE") - multi.intergration.GLOBAL["__SYSTEMBENCHMARK__"]=n - local sThread=multi.intergration.THREAD - local GLOBAL=multi.intergration.GLOBAL + multi.integration.GLOBAL["__SYSTEMBENCHMARK__"]=n + local sThread=multi.integration.THREAD + local GLOBAL=multi.integration.GLOBAL for i=1,cores do multi:newSystemThread("STHREAD_BENCH",function() require("multi") @@ -170,7 +179,130 @@ function multi:newSystemThreadedTable(name) }) return self end - multi.intergration.GLOBAL[name]=c -- send the object to the thread through the global interface + multi.integration.GLOBAL[name]=c -- send the object to the thread through the global interface end return c end +function multi:newSystemThreadedJobQueue(numOfCores) + local c={} + c.jobnum=1 + c.cores=numOfCores or multi.integration.THREAD.getCores() + c.queueIN=multi:newSystemThreadedQueue("THREADED_JQ"):init() + c.queueOUT=multi:newSystemThreadedQueue("THREADED_JQO"):init() + c.REG=multi:newSystemThreadedTable("THREADED_JQ_F_REG"):init() + -- registerJob(name,func) + -- pushJob(...) + function c:registerJob(name,func) + self.REG[name]=func + end + function c:pushJob(name,...) + self.queueOUT:push({self.jobnum,name,...}) + self.jobnum=self.jobnum+1 + end + local GLOBAL=multi.integration.GLOBAL -- set up locals incase we are using lanes + local sThread=multi.integration.THREAD -- set up locals incase we are using lanes + GLOBAL["__JQ_COUNT__"]=c.cores + for i=1,c.cores do + multi:newSystemThread("System Threaded Job Queue Worker Thread #"..i,function() + require("multi") + __sleep__=.001 + if love then -- lets make sure we don't reference upvalues if using love2d + GLOBAL=_G.GLOBAL + sThread=_G.sThread + __sleep__=.1 + end + JQI=sThread.waitFor("THREADED_JQO"):init() -- Grab it + JQO=sThread.waitFor("THREADED_JQ"):init() -- Grab it + FGLOBAL=sThread.waitFor("THREADED_JQ_F_REG"):init() -- Grab it + sThread.sleep(.1) -- lets wait for things to work out + setmetatable(_G,{ + __index=FGLOBAL + }) + GLOBAL["THREADED_JQ"]=nil -- remove it + GLOBAL["THREADED_JQO"]=nil -- remove it + GLOBAL["THREADED_JQ_F_REG"]=nil -- remove it + multi:newLoop(function() + sThread.sleep(__sleep__) -- lets allow cpu time for other processes on our system! + local job=JQI:pop() + if job then + local ID=table.remove(job,1) -- return and remove + local name=table.remove(job,1) -- return and remove + local ret={FGLOBAL:waitFor(name)(unpack(job))} -- unpack the rest + JQO:push({ID,ret}) + end + end) + multi:mainloop() + end) + end + c.OnJobCompleted=multi:newConnection() + c.updater=multi:newLoop(function(self) + local data=self.link.queueIN:pop() + while data do + if data then + self.link.OnJobCompleted:Fire(unpack(data)) + end + data=self.link.queueIN:pop() + end + end) + c.updater.link=c + return c +end +if love then + if love.thread then + function multi:newSystemThreadedJobQueue(numOfCores) + local c={} + c.jobnum=1 + c.cores=numOfCores or multi.integration.THREAD.getCores() + c.queueIN=multi:newSystemThreadedQueue("THREADED_JQ"):init() + c.queueOUT=multi:newSystemThreadedQueue("THREADED_JQO"):init() + function c:registerJob(name,func) + GLOBAL["__TJQ__"..name.."__"]=func + end + function c:pushJob(name,...) + self.queueOUT:push({self.jobnum,name,...}) + self.jobnum=self.jobnum+1 + end + local GLOBAL=multi.integration.GLOBAL -- set up locals incase we are using lanes + local sThread=multi.integration.THREAD -- set up locals incase we are using lanes + GLOBAL["__JQ_COUNT__"]=c.cores + for i=1,c.cores do + multi:newSystemThread("System Threaded Job Queue Worker Thread #"..i,function() + GLOBAL=_G.GLOBAL + sThread=_G.sThread + local JQI=sThread.waitFor("THREADED_JQO"):init() -- Grab it + local JQO=sThread.waitFor("THREADED_JQ"):init() -- Grab it + sThread.sleep(.1) -- lets wait for things to work out + setmetatable(_G,{ + __index=function(t,k,v) + return GLOBAL["__TJQ__"..k.."__"] + end + }) + GLOBAL["THREADED_JQ"]=nil -- remove it + GLOBAL["THREADED_JQO"]=nil -- remove it + multi:newLoop(function() + sThread.sleep(.001) -- lets allow cpu time for other processes on our system! + local job=JQI:pop() + if job then + local ID=table.remove(job,1) -- return and remove + local name=table.remove(job,1) -- return and remove + local ret={sThread.waitFor("__TJQ__"..name.."__")(unpack(job))} -- unpack the rest + JQO:push({ID,ret}) + end + end) + end) + end + c.OnJobCompleted=multi:newConnection() + c.updater=multi:newLoop(function(self) + local data=self.link.queueIN:pop() + while data do + if data then + self.link.OnJobCompleted:Fire(unpack(data)) + end + data=self.link.queueIN:pop() + end + end) + c.updater.link=c + return c + end + end +end diff --git a/multi/integration/loveManager.lua b/multi/integration/loveManager.lua index e357732..0a144d9 100644 --- a/multi/integration/loveManager.lua +++ b/multi/integration/loveManager.lua @@ -29,6 +29,7 @@ setmetatable(GLOBAL,{ function __sync__() local data=__mythread__:pop() while data do + love.timer.sleep(.001) if type(data)=="string" then local cmd,tp,name,d=data:match("(%S-) (%S-) (%S-) (.+)") if name=="__DIEPLZ"..__THREADNAME__.."__" then @@ -153,23 +154,7 @@ function sThread.hold(n) repeat __sync__() until n() end updater=multi:newUpdater() -updater:OnUpdate(function(self) - local data=__mythread__:pop() - while data do - if type(data)=="string" then - local cmd,tp,name,d=data:match("(%S-) (%S-) (%S-) (.+)") - if name=="__DIEPLZ"..__THREADNAME__.."__" then - error("Thread: "..__THREADNAME__.." has been stopped!") - end - if cmd=="SYNC" then - __proxy__[name]=resolveType(tp,d) - end - else - __proxy__[name]=data - end - data=__mythread__:pop() - end -end) +updater:OnUpdate(__sync__) func=loadDump([=[INSERT_USER_CODE]=])() multi:mainloop() ]] -- 2.43.0 From 112e20f78898944c80c0652d345ef4c105aba194 Mon Sep 17 00:00:00 2001 From: Ryan Ward Date: Sun, 2 Jul 2017 23:19:09 -0400 Subject: [PATCH 3/5] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 58ab872..7d133a2 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # multi Version: 1.8.4 (System Threaded Job Queues) -**Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme?**
    +**Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme?** Also I added a Testing Branch. That Branch will have the latest updates, but those updates may be unstable. I like to keep the master as bug free as possible
    My multitasking library for lua
    To install copy the multi folder into your enviroment and you are good to go
    -- 2.43.0 From 01b5f865f9edde31f5d86d02a62990eac8fc2006 Mon Sep 17 00:00:00 2001 From: Ryan Date: Sun, 2 Jul 2017 23:22:35 -0400 Subject: [PATCH 4/5] updated share.lua --- multi/integration/shared/shared.lua | 59 +++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/multi/integration/shared/shared.lua b/multi/integration/shared/shared.lua index b393612..06bb8cf 100644 --- a/multi/integration/shared/shared.lua +++ b/multi/integration/shared/shared.lua @@ -247,3 +247,62 @@ function multi:newSystemThreadedJobQueue(numOfCores) c.updater.link=c return c end +if love then + if love.thread then + function multi:newSystemThreadedJobQueue(numOfCores) + local c={} + c.jobnum=1 + c.cores=numOfCores or multi.integration.THREAD.getCores() + c.queueIN=multi:newSystemThreadedQueue("THREADED_JQ"):init() + c.queueOUT=multi:newSystemThreadedQueue("THREADED_JQO"):init() + function c:registerJob(name,func) + GLOBAL["__TJQ__"..name.."__"]=func + end + function c:pushJob(name,...) + self.queueOUT:push({self.jobnum,name,...}) + self.jobnum=self.jobnum+1 + end + local GLOBAL=multi.integration.GLOBAL -- set up locals incase we are using lanes + local sThread=multi.integration.THREAD -- set up locals incase we are using lanes + GLOBAL["__JQ_COUNT__"]=c.cores + for i=1,c.cores do + multi:newSystemThread("System Threaded Job Queue Worker Thread #"..i,function() + GLOBAL=_G.GLOBAL + sThread=_G.sThread + local JQI=sThread.waitFor("THREADED_JQO"):init() -- Grab it + local JQO=sThread.waitFor("THREADED_JQ"):init() -- Grab it + sThread.sleep(.1) -- lets wait for things to work out + setmetatable(_G,{ + __index=function(t,k,v) + return GLOBAL["__TJQ__"..k.."__"] + end + }) + GLOBAL["THREADED_JQ"]=nil -- remove it + GLOBAL["THREADED_JQO"]=nil -- remove it + multi:newLoop(function() + sThread.sleep(.001) -- lets allow cpu time for other processes on our system! + local job=JQI:pop() + if job then + local ID=table.remove(job,1) -- return and remove + local name=table.remove(job,1) -- return and remove + local ret={sThread.waitFor("__TJQ__"..name.."__")(unpack(job))} -- unpack the rest + JQO:push({ID,ret}) + end + end) + end) + end + c.OnJobCompleted=multi:newConnection() + c.updater=multi:newLoop(function(self) + local data=self.link.queueIN:pop() + while data do + if data then + self.link.OnJobCompleted:Fire(unpack(data)) + end + data=self.link.queueIN:pop() + end + end) + c.updater.link=c + return c + end + end +end -- 2.43.0 From 209f833b3f690783b5563c5473c35f990bac3fd8 Mon Sep 17 00:00:00 2001 From: Ryan Date: Mon, 3 Jul 2017 13:54:32 -0400 Subject: [PATCH 5/5] (1.8.4) Update --- .gitignore | 1 + README.html | 185 ++++++++++++++++-- README.md | 102 ++++++++-- examples/lanesintegratetest7.lua | 16 +- examples/love2d Threading Example/main5.lua | 21 -- .../love2d Threading Example/multi/init.lua | 122 +++++++++++- .../multi/integration/lanesManager.lua | 13 +- .../multi/integration/loveManager.lua | 73 ++++--- .../multi/integration/shared/shared.lua | 1 - multi/init.lua | 8 +- multi/integration/loveManager.lua | 8 + multi/integration/shared/shared.lua | 1 - 12 files changed, 444 insertions(+), 107 deletions(-) diff --git a/.gitignore b/.gitignore index 3847865..c11b7e0 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ test.lua +test2.lua diff --git a/README.html b/README.html index 4dd3bb4..c461431 100644 --- a/README.html +++ b/README.html @@ -9,7 +9,8 @@ -

    multi Version: 1.8.3 (Performance Increase! Check changes for info)

    Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme?

    My multitasking library for lua

    To install copy the multi folder into your enviroment and you are good to go

    It is a pure lua binding if you ingore the integrations and the love2d compat

    If you find any bugs or have any issues please let me know :)

      +

      multi Version: 1.8.4 (System Threaded Job Queues)

      Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme? Also I added a Testing Branch. That Branch will have the latest updates, but those updates may be unstable. I like to keep the master as bug free as possible

      In Changes you’ll find documentation for(In Order):

        +
      • System Threaded Job Queues
      • New mainloop functions
      • System Threaded Tables
      • System Threaded Benchmark
      • System Threaded Queues
      • Threading related features
      • And backwards compat stuff

      My multitasking library for lua

      To install copy the multi folder into your enviroment and you are good to go

      It is a pure lua binding if you ingore the integrations and the love2d compat

      If you find any bugs or have any issues please let me know :)

      If you don’t see a table of contents try using the ReadMe.html file. It is eaiser to navigate the readme

      • @@ -186,7 +292,7 @@

        INSTALLING

        luarocks install multi
         

        Discord

        For real-time assistance with my libraries! A place where you can ask questions and get help with any of my libraries. Also you can request features and stuff there as well.

        https://discord.gg/U8UspuA

        Planned features/TODO

          -
        • Add system threads for love2d that works like the lanesManager (loveManager, slight differences).
        • Improve performance of the library
        • Improve coroutine based threading scheduling
        • Add more features to support module creators
        • Make a framework for eaiser thread task distributing
        • Fix Error handling on threaded multi objects Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance!
        • Add multi:OnError(function(obj,err))
        • sThread.wrap(obj) May or may not be completed Theory: Allows interaction in one thread to affect it in another. The addition to threaded tables may make this possible!
        • SystemThreaded Actors — After some tests i figured out a way to make this work… It will work slightly different though. This is due to the actor needing to be splittable…
        • LoadBalancing for system threads (Once SystemThreaded Actors are done)
        • Add more integrations
        • Finish the wiki stuff. (11% done)
        • Test for unknown bugs

        Known Bugs/Issues

        In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P

        Usage:

         Add system threads for love2d that works like the lanesManager (loveManager, slight differences).
      • Improve performance of the library
      • Improve coroutine based threading scheduling
      • Improve love2d Idle thread cpu usage… Tricky Look at the rambling section for insight.
      • Add more features to support module creators
      • Make a framework for eaiser thread task distributing
      • Fix Error handling on threaded multi objects Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance!
      • Add multi:OnError(function(obj,err))
      • sThread.wrap(obj) May or may not be completed Theory: Allows interaction in one thread to affect it in another. The addition to threaded tables may make this possible!
      • SystemThreaded Actors — After some tests i figured out a way to make this work… It will work slightly different though. This is due to the actor needing to be splittable…
      • LoadBalancing for system threads (Once SystemThreaded Actors are done)
      • Add more integrations
      • Finish the wiki stuff. (11% done)
      • Test for unknown bugs

      Known Bugs/Issues

      In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P

      Another bug concerns the SystemThreadedJobQueue, Only 1 can be used for now… Vreating more may not be a good idea.

      Usage:

      function<
           end
       end)
       multi:mainloop()
      -

      Output (Change the value inc as indicated in the comment to see the outcomes!)

      Looping…

      Looping…

      Looping…

      Looping…

      Looping…

      Looping…

      Looping…

      Looping…

      Looping…

      Loop timed out! tloop Trying again…

      Looping…

      Looping…

      Looping…

      Looping…

      Looping…

      We did it! 1 2 3

      Changes

      Updated from 1.8.2 to 1.8.3

      Added:

      New Mainloop functions Below you can see the slight differences… Function overhead is not too bad in lua, but has a real difference. multi:mainloop() and multi:unprotectedMainloop() use the same algorithm yet the dedicated unprotected one is slightly faster due to having less function overhead.

        -
      • multi:mainloop()* — Bench: 16830003 Steps in 3 second(s)!
      • multi:protectedMainloop() — Bench: 16699308 Steps in 3 second(s)!
      • multi:unprotectedMainloop() — Bench: 16976627 Steps in 3 second(s)!
      • multi:prioritizedMainloop1() — Bench: 15007133 Steps in 3 second(s)!
      • multi:prioritizedMainloop2() — Bench: 15526248 Steps in 3 second(s)!

      * The OG mainloop function remains the same and old methods to achieve what we have with the new ones still exist

      These new methods help by removing function overhead that is caused through the original mainloop function. The one downside is that you no longer have the flexiblity to change the processing during runtime.

      However there is a work around! You can use processes to run multiobjs as well and use the other methods on them.

      I may make a full comparison between each method and which is faster, but for now trust that the dedicated ones with less function overhead are infact faster. Not by much but still faster. :D
      Updated from 1.8.1 to 1.8.2

      Added:

        +

    Output (Change the value inc as indicated in the comment to see the outcomes!)

    Looping…

    Looping…

    Looping…

    Looping…

    Looping…

    Looping…

    Looping…

    Looping…

    Looping…

    Loop timed out! tloop Trying again…

    Looping…

    Looping…

    Looping…

    Looping…

    Looping…

    We did it! 1 2 3

    Changes

    Updated from 1.8.3 to 1.8.4

    Added:

      +
    • multi:newSystemThreadedJobQueue()
    • Improved stability of the library
    • Fixed a bug that made the benchmark and getload commands non-thread(coroutine) safe
    • Tweaked the loveManager to help improve idle cpu usage
    • Minor tweaks to the coroutine scheduling

    Using multi:newSystemThreadedJobQueue()

    First you need to create the object
    This works the same way as love2d as it does with lanes… It is getting increasing harder to make both work the same way with speed in mind… Anyway…

    -- Creating the object using lanes manager to show case this. Examples has the file for love2d
    +local GLOBAL,sThread=require("multi.integration.lanesManager").init()
    +jQueue=multi:newSystemThreadedJobQueue(n) -- this internally creates System threads. By defualt it will use the # of processors on your system You can set this number though.
    +-- Only create 1 jobqueue! For now making more than 1 is buggy. You only really need one though. Just register new functions if you want 1 queue to do more. The one reason though is keeping track of jobIDs. I have an idea that I will roll out in the next update.
    +jQueue:registerJob("TEST_JOB",function(a,s)
    +    math.randomseed(s)
    +    -- We will push a random #
    +    TEST_JOB2() -- You can call other registered functions as well!
    +    return math.random(0,255) -- send the result to the main thread
    +end)
    +jQueue:registerJob("TEST_JOB2",function()
    +    print("Test Works!") -- this is called from the job since it is registered on the same queue
    +end)
    +tableOfOrder={} -- This is how we will keep order of our completed jobs. There is no guarantee that the order will be correct
    +jQueue.OnJobCompleted(function(JOBID,n) -- whenever a job is completed you hook to the event that is called. This passes the JOBID folled by the returns of the job
    +    -- JOBID is the completed job, starts at 1 and counts up by 1.
    +    -- Threads finish at different times so jobids may be passed out of order! Be sure to have a way to order them
    +    tableOfOrder[JOBID]=n -- we order ours by putting them into a table
    +    if #tableOfOrder==10 then
    +        print("We got all of the pieces!")
    +    end
    +end)
    +-- Lets push the jobs now
    +for i=1,10 do -- Job Name of registered function, ... varargs
    +    jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000))
    +end
    +print("I pushed all of the jobs :)")
    +multi:mainloop() -- Start the main loop :D
    +

    Thats it from this version!

    Updated from 1.8.2 to 1.8.3

    Added:

    New Mainloop functions Below you can see the slight differences… Function overhead is not too bad in lua, but has a real difference. multi:mainloop() and multi:unprotectedMainloop() use the same algorithm yet the dedicated unprotected one is slightly faster due to having less function overhead.

      +
    • multi:mainloop()* — Bench: 16830003 Steps in 3 second(s)!
    • multi:protectedMainloop() — Bench: 16699308 Steps in 3 second(s)!
    • multi:unprotectedMainloop() — Bench: 16976627 Steps in 3 second(s)!
    • multi:prioritizedMainloop1() — Bench: 15007133 Steps in 3 second(s)!
    • multi:prioritizedMainloop2() — Bench: 15526248 Steps in 3 second(s)!

    * The OG mainloop function remains the same and old methods to achieve what we have with the new ones still exist

    These new methods help by removing function overhead that is caused through the original mainloop function. The one downside is that you no longer have the flexiblity to change the processing during runtime.

    However there is a work around! You can use processes to run multiobjs as well and use the other methods on them.

    I may make a full comparison between each method and which is faster, but for now trust that the dedicated ones with less function overhead are infact faster. Not by much but still faster. :D

    Updated from 1.8.1 to 1.8.2

    Added:

    • multi:newsystemThreadedTable(name) NOTE: Metatables are not supported in transfers. However there is a work around obj:init() that you see does this. Take a look in the multi/integration/shared/shared.lua files to see how I did it!
    • Modified the GLOBAL metatable to sync before doing its tests
    • multi._VERSION was multi.Version, felt it would be more consistant this way… I left the old way of getting the version just incase someone has used that way. It will eventually be gone. Also multi:getVersion() will do the job just as well and keep your code nice and update related bug free!
    • Also everything that is included in the: multi/integration/shared/shared.lua (Which is loaded automatically) works in both lanes and love2d enviroments!

    The threaded table is setup just like the threaded queue.

    It provids GLOBAL like features without having to write to GLOBAL!

    This is useful for module creators who want to keep their data private, but also use GLOBAL like coding.

    It has a few features that makes it a bit better than plain ol GLOBAL (For now…)
    (ThreadedTable - TT for short)

    • TT:waitFor(name)
    • TT:sync()
    • TT[“var”]=value
    • print(TT[“var”])

    we also have the “sync” method, this one was made for love2d because we do a syncing trick to get data in a table format. The lanes side has a sync method as well so no worries. Using indexing calls sync once and may grab your variable. This allows you to have the lanes indexing ‘like’ syntax when doing regular indexing in love2d side of the module. As of right now both sides work flawlessly! And this effect is now the GLOBAL as well

    On GLOBALS sync is a internal method for keeping the GLOBAL table in order. You can still use sThread.waitFor(name) to wait for variables that may of may not yet exist!

    Time for some examples:

    Using multi:newSystemThreadedTable(name)

    "test2","no done yet!",0,0,300,100)
     t:centerX()
     t:centerY()
    -

    Updated from 1.8.0 to 1.8.1

    No real change!

    Changed the structure of the library. Combined the coroutine based threads into the core!

    Only compat and integrations are not part of the core and never will be by nature.

    This should make the library more convient to use.

    I left multi/all.lua file so if anyone had libraries/projects that used that it will still work!

    Updated from 1.7.6 to 1.8.0
    (How much thread could a thread thread if a thread could thread thread?)
    Added:

      +

    Updated from 1.8.0 to 1.8.1

    No real change!

    Changed the structure of the library. Combined the coroutine based threads into the core!

    Only compat and integrations are not part of the core and never will be by nature.

    This should make the library more convient to use.

    I left multi/all.lua file so if anyone had libraries/projects that used that it will still work!

    Updated from 1.7.6 to 1.8.0
    (How much thread could a thread thread if a thread could thread thread?)
    Added:

    • multi:newSystemThreadedQueue()
    • multi:systemThreadedBenchmark()
    • More example files
    • multi:canSystemThread() — true if an integration was added false otherwise (For module creation)
    • Fixed a few bugs in the loveManager

    Using multi:systemThreadedBenchmark()

    "test!",end
     end)
     multi:mainloop()
    -

    Updated from 1.7.5 to 1.7.6

    Fixed:
    Typos like always
    Added:

    multi:getPlatform() — returns “love2d” if using the love2d platform or returns “lanes” if using lanes for threading

    examples files

    In Events added method setTask(func)

    The old way still works and is more convient to be honest, but I felt a method to do this was ok.

    Updated:
    some example files to reflect changes to the core. Changes allow for less typing

    loveManager to require the compat if used so you don’t need 2 require line to retrieve the library

    Updated from 1.7.4 to 1.7.5

    Fixed some typos in the readme… (I am sure there are more there are always more)

    Added more features for module support

    TODO:

    Work on performance of the library… I see 3 places where I can make this thing run quicker

    I’ll show case some old versions of the multitasking library eventually so you can see its changes in days past!

    Updated from 1.7.3 to 1.7.4

    Added: the example folder which will be populated with more examples in the near future!

    The loveManager integration that mimics the lanesManager integration almost exactly to keep coding in both enviroments as close to possible. This is done mostly for library creation support!

    An example of the loveManager in action using almost the same code as the lanesintergreationtest2.lua

    NOTE: This code has only been tested to work on love2d version 1.10.2 thoough it should work version 0.9.0

    Updated from 1.7.5 to 1.7.6

    Fixed:
    Typos like always
    Added:

    multi:getPlatform() — returns “love2d” if using the love2d platform or returns “lanes” if using lanes for threading

    examples files

    In Events added method setTask(func)

    The old way still works and is more convient to be honest, but I felt a method to do this was ok.

    Updated:
    some example files to reflect changes to the core. Changes allow for less typing

    loveManager to require the compat if used so you don’t need 2 require line to retrieve the library

    Updated from 1.7.4 to 1.7.5

    Fixed some typos in the readme… (I am sure there are more there are always more)

    Added more features for module support

    TODO:

    Work on performance of the library… I see 3 places where I can make this thing run quicker

    I’ll show case some old versions of the multitasking library eventually so you can see its changes in days past!

    Updated from 1.7.3 to 1.7.4

    Added: the example folder which will be populated with more examples in the near future!

    The loveManager integration that mimics the lanesManager integration almost exactly to keep coding in both enviroments as close to possible. This is done mostly for library creation support!

    An example of the loveManager in action using almost the same code as the lanesintergreationtest2.lua

    NOTE: This code has only been tested to work on love2d version 1.10.2 thoough it should work version 0.9.0

    "Bench"]=3"no done yet!",0,0,300,100)
     t:centerX()
     t:centerY()
    -

    Updated from 1.7.2 to 1.7.3

    Changed how requiring the library works!
    require("multi.all") Will still work as expected; however, with the exception of threading, compat, and integrations everything else has been moved into the core of the library.

    Updated from 1.7.2 to 1.7.3

    Changed how requiring the library works!
    require("multi.all") Will still work as expected; however, with the exception of threading, compat, and integrations everything else has been moved into the core of the library.

    require("multi.step")
     require("multi.task")
     -- ^ they are all part of the core now
    -

    Updated from 1.7.1 to 1.7.2

    Moved updaters, loops, and alarms into the init.lua file. I consider them core features and they are referenced in the init.lua file so they need to exist there. Threaded versions are still separate though. Added another example file

    Updated from 1.7.0 to 1.7.1 Bug fixes only

    Updated from 1.6.0 to 1.7.0

    Modified: multi.integration.lanesManager.lua
    It is now in a stable and simple state Works with the latest lanes version! Tested with version 3.11 I cannot promise that everything will work with eariler versions. Future versions are good though.

    Example Usage:

    sThread is a handle to a global interface for system threads to interact with themself

    thread is the interface for multithreads as seen in the threading section

    GLOBAL a table that can be used throughout each and every thread

    sThreads have a few methods

    sThread.set(name,val) — you can use the GLOBAL table instead modifies the same table anyway

    sThread.get(name) — you can use the GLOBAL table instead modifies the same table anyway

    sThread.waitFor(name) — waits until a value exists, if it does it returns it

    sThread.getCores() — returns the number of cores on your cpu

    sThread.sleep(n) — sleeps for a bit stopping the entire thread from running

    sThread.hold(n) — sleeps until a condition is met

    Updated from 1.7.1 to 1.7.2

    Moved updaters, loops, and alarms into the init.lua file. I consider them core features and they are referenced in the init.lua file so they need to exist there. Threaded versions are still separate though. Added another example file

    Updated from 1.7.0 to 1.7.1 Bug fixes only

    Updated from 1.6.0 to 1.7.0

    Modified: multi.integration.lanesManager.lua
    It is now in a stable and simple state Works with the latest lanes version! Tested with version 3.11 I cannot promise that everything will work with eariler versions. Future versions are good though.

    Example Usage:

    sThread is a handle to a global interface for system threads to interact with themself

    thread is the interface for multithreads as seen in the threading section

    GLOBAL a table that can be used throughout each and every thread

    sThreads have a few methods

    sThread.set(name,val) — you can use the GLOBAL table instead modifies the same table anyway

    sThread.get(name) — you can use the GLOBAL table instead modifies the same table anyway

    sThread.waitFor(name) — waits until a value exists, if it does it returns it

    sThread.getCores() — returns the number of cores on your cpu

    sThread.sleep(n) — sleeps for a bit stopping the entire thread from running

    sThread.hold(n) — sleeps until a condition is met

    "test",end)
     multi:mainloop()
    -

    Updated from 1.5.0 to 1.6.0

    Changed: steps and loops

    Updated from 1.5.0 to 1.6.0

    Changed: steps and loops

    function
    ">require("multi.all") require("multi.compat.backwards[1,5,0]") -- allows for the use of features that were scrapped/changed in 1.6.0+ -

    Updated from 1.4.1 to 1.5.0
    Added:

      -
    • An easy way to manage timeouts
    • Small bug fixes

    1.4.1 - First Public release of the library

    IMPORTANT:

    Every update I make aims to make things simpler more efficent and just better, but a lot of old code, which can be really big, uses a lot of older features. I know the pain of having to rewrite everything. My promise to my library users is that I will always have backwards support for older features! New ways may exist that are quicker and eaiser, but the old features/methods will be supported.

    +

    Updated from 1.4.1 to 1.5.0

    Added:

      +
    • An easy way to manage timeouts
    • Small bug fixes

    1.4.1 - First Public release of the library

    IMPORTANT:

    Every update I make aims to make things simpler more efficent and just better, but a lot of old code, which can be really big, uses a lot of older features. I know the pain of having to rewrite everything. My promise to my library users is that I will always have backwards support for older features! New ways may exist that are quicker and eaiser, but the old features/methods will be supported.

    Rambling

    Love2d: Sleeping reduces the cpu time making my load detection think the system is under more load, thus preventing it from sleeping… I will look into other means. As of right now it will not eat all of your cpu if threads are active. For now I suggest killing threads that aren’t needed anymore. On lanes threads at idle use 0% cpu and it is amazing. A state machine may solve what I need though. One state being idle state that sleeps and only goes into the active state if a job request or data is sent to it… after some time of not being under load it wil switch back into the idle state… We’ll see what happens.

    Love2d doesn’t like to send functions through channels. By defualt it does not support this. I achieve this by dumping the function and loadstring it on the thread. This however is slow. For the System Threaded Job Queue I had to change my original idea of sending functions as jobs. The current way you do it now is register a job functions once and then call that job across the thread through a queue. Each worker thread pops from the queue and returns the job. The Job ID is automatically updated and allows you to keep track of the order that the data comes in. A table with # indexes can be used to originze the data…

    In regards to benchmarking. If you see my bench marks and are wondering they are 10x better its because I am using luajit for my tests. I highly recommend using luajit for my library, but lua 5.1 will work just as well, but not as fast.

    diff --git a/README.md b/README.md index 7d133a2..f231ca5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,15 @@ # multi Version: 1.8.4 (System Threaded Job Queues) **Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme?** Also I added a Testing Branch. That Branch will have the latest updates, but those updates may be unstable. I like to keep the master as bug free as possible
    +In Changes you'll find documentation for(In Order): +- System Threaded Job Queues +- New mainloop functions +- System Threaded Tables +- System Threaded Benchmark +- System Threaded Queues +- Threading related features +- And backwards compat stuff + My multitasking library for lua
    To install copy the multi folder into your enviroment and you are good to go
    @@ -8,6 +17,8 @@ It is a pure lua binding if you ingore the integrations and the love2d compat + [TOC] INSTALLING @@ -24,7 +35,8 @@ Planned features/TODO --------------------- - [x] ~~Add system threads for love2d that works like the lanesManager (loveManager, slight differences).~~ - [x] ~~Improve performance of the library~~ -- [ ] Improve coroutine based threading scheduling +- [x] ~~Improve coroutine based threading scheduling~~ +- [ ] Improve love2d Idle thread cpu usage... Tricky Look at the rambling section for insight. - [x] ~~Add more features to support module creators~~ - [x] ~~Make a framework for eaiser thread task distributing~~ - [x] ~~Fix Error handling on threaded multi objects~~ Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance! @@ -40,6 +52,8 @@ Known Bugs/Issues ----------------- In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to...) not use the multi library within the thread. A fix for this is to call `multi:Stop()` when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data... You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P +Another bug concerns the SystemThreadedJobQueue, Only 1 can be used for now... Vreating more may not be a good idea. + Usage:
    ----- ```lua @@ -785,7 +799,53 @@ We did it! 1 2 3
    Changes ------- -Updated from 1.8.2 to 1.8.3
    +Updated from 1.8.3 to 1.8.4 +--------------------------- +Added: +- multi:newSystemThreadedJobQueue() +- Improved stability of the library +- Fixed a bug that made the benchmark and getload commands non-thread(coroutine) safe +- Tweaked the loveManager to help improve idle cpu usage +- Minor tweaks to the coroutine scheduling + +# Using multi:newSystemThreadedJobQueue() +First you need to create the object +This works the same way as love2d as it does with lanes... It is getting increasing harder to make both work the same way with speed in mind... Anyway... +```lua +-- Creating the object using lanes manager to show case this. Examples has the file for love2d +local GLOBAL,sThread=require("multi.integration.lanesManager").init() +jQueue=multi:newSystemThreadedJobQueue(n) -- this internally creates System threads. By defualt it will use the # of processors on your system You can set this number though. +-- Only create 1 jobqueue! For now making more than 1 is buggy. You only really need one though. Just register new functions if you want 1 queue to do more. The one reason though is keeping track of jobIDs. I have an idea that I will roll out in the next update. +jQueue:registerJob("TEST_JOB",function(a,s) + math.randomseed(s) + -- We will push a random # + TEST_JOB2() -- You can call other registered functions as well! + return math.random(0,255) -- send the result to the main thread +end) +jQueue:registerJob("TEST_JOB2",function() + print("Test Works!") -- this is called from the job since it is registered on the same queue +end) +tableOfOrder={} -- This is how we will keep order of our completed jobs. There is no guarantee that the order will be correct +jQueue.OnJobCompleted(function(JOBID,n) -- whenever a job is completed you hook to the event that is called. This passes the JOBID folled by the returns of the job + -- JOBID is the completed job, starts at 1 and counts up by 1. + -- Threads finish at different times so jobids may be passed out of order! Be sure to have a way to order them + tableOfOrder[JOBID]=n -- we order ours by putting them into a table + if #tableOfOrder==10 then + print("We got all of the pieces!") + end +end) +-- Lets push the jobs now +for i=1,10 do -- Job Name of registered function, ... varargs + jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000)) +end +print("I pushed all of the jobs :)") +multi:mainloop() -- Start the main loop :D +``` + +Thats it from this version! + +Updated from 1.8.2 to 1.8.3 +--------------------------- Added:
    **New Mainloop functions** Below you can see the slight differences... Function overhead is not too bad in lua, but has a real difference. multi:mainloop() and multi:unprotectedMainloop() use the same algorithm yet the dedicated unprotected one is slightly faster due to having less function overhead. - multi:mainloop()\* -- Bench: 16830003 Steps in 3 second(s)! @@ -801,7 +861,9 @@ These new methods help by removing function overhead that is caused through the However there is a work around! You can use processes to run multiobjs as well and use the other methods on them. I may make a full comparison between each method and which is faster, but for now trust that the dedicated ones with less function overhead are infact faster. Not by much but still faster. :D -Updated from 1.8.1 to 1.8.2
    + +Updated from 1.8.1 to 1.8.2 +--------------------------- Added:
    - multi:newsystemThreadedTable(name) NOTE: Metatables are not supported in transfers. However there is a work around obj:init() that you see does this. Take a look in the multi/integration/shared/shared.lua files to see how I did it! - Modified the GLOBAL metatable to sync before doing its tests @@ -864,7 +926,8 @@ t:centerX() t:centerY() ``` -Updated from 1.8.0 to 1.8.1
    +Updated from 1.8.0 to 1.8.1 +--------------------------- No real change!
    Changed the structure of the library. Combined the coroutine based threads into the core!
    Only compat and integrations are not part of the core and never will be by nature.
    @@ -991,7 +1054,8 @@ multi:newThread("test!",function() -- this is a lua thread end) multi:mainloop() ``` -Updated from 1.7.5 to 1.7.6
    +Updated from 1.7.5 to 1.7.6 +--------------------------- Fixed: Typos like always Added:
    @@ -1003,7 +1067,8 @@ The old way still works and is more convient to be honest, but I felt a method t Updated: some example files to reflect changes to the core. Changes allow for less typing
    loveManager to require the compat if used so you don't need 2 require line to retrieve the library
    -Updated from 1.7.4 to 1.7.5
    +Updated from 1.7.4 to 1.7.5 +--------------------------- Fixed some typos in the readme... (I am sure there are more there are always more)
    Added more features for module support
    TODO:
    @@ -1011,7 +1076,8 @@ Work on performance of the library... I see 3 places where I can make this thing I'll show case some old versions of the multitasking library eventually so you can see its changes in days past!
    -Updated from 1.7.3 to 1.7.4
    +Updated from 1.7.3 to 1.7.4 +--------------------------- Added: the example folder which will be populated with more examples in the near future!
    The loveManager integration that mimics the lanesManager integration almost exactly to keep coding in both enviroments as close to possible. This is done mostly for library creation support!
    An example of the loveManager in action using almost the same code as the lanesintergreationtest2.lua
    @@ -1088,7 +1154,8 @@ t=gui:newTextLabel("no done yet!",0,0,300,100) t:centerX() t:centerY() ``` -Updated from 1.7.2 to 1.7.3
    +Updated from 1.7.2 to 1.7.3 +--------------------------- Changed how requiring the library works! `require("multi.all")` Will still work as expected; however, with the exception of threading, compat, and integrations everything else has been moved into the core of the library. ```lua @@ -1104,12 +1171,15 @@ require("multi.task") -- ^ they are all part of the core now ``` -Updated from 1.7.1 to 1.7.2
    +Updated from 1.7.1 to 1.7.2 +--------------------------- Moved updaters, loops, and alarms into the init.lua file. I consider them core features and they are referenced in the init.lua file so they need to exist there. Threaded versions are still separate though. Added another example file Updated from 1.7.0 to 1.7.1 Bug fixes only +--------------------------- -Updated from 1.6.0 to 1.7.0
    +Updated from 1.6.0 to 1.7.0 +--------------------------- Modified: multi.integration.lanesManager.lua It is now in a stable and simple state Works with the latest lanes version! Tested with version 3.11 I cannot promise that everything will work with eariler versions. Future versions are good though.
    Example Usage:
    @@ -1162,7 +1232,8 @@ end) multi:mainloop() ``` -Updated from 1.5.0 to 1.6.0
    +Updated from 1.5.0 to 1.6.0 +--------------------------- Changed: steps and loops ```lua -- Was @@ -1186,11 +1257,20 @@ require("multi.all") require("multi.compat.backwards[1,5,0]") -- allows for the use of features that were scrapped/changed in 1.6.0+ ``` Updated from 1.4.1 to 1.5.0 +--------------------------- Added: - An easy way to manage timeouts - Small bug fixes 1.4.1 - First Public release of the library +--------------------------- IMPORTANT:
    Every update I make aims to make things simpler more efficent and just better, but a lot of old code, which can be really big, uses a lot of older features. I know the pain of having to rewrite everything. My promise to my library users is that I will always have backwards support for older features! New ways may exist that are quicker and eaiser, but the old features/methods will be supported.
    +Rambling +-------- +Love2d: Sleeping reduces the cpu time making my load detection think the system is under more load, thus preventing it from sleeping... I will look into other means. As of right now it will not eat all of your cpu if threads are active. For now I suggest killing threads that aren't needed anymore. On lanes threads at idle use 0% cpu and it is amazing. A state machine may solve what I need though. One state being idle state that sleeps and only goes into the active state if a job request or data is sent to it... after some time of not being under load it wil switch back into the idle state... We'll see what happens. + +Love2d doesn't like to send functions through channels. By defualt it does not support this. I achieve this by dumping the function and loadstring it on the thread. This however is slow. For the System Threaded Job Queue I had to change my original idea of sending functions as jobs. The current way you do it now is register a job functions once and then call that job across the thread through a queue. Each worker thread pops from the queue and returns the job. The Job ID is automatically updated and allows you to keep track of the order that the data comes in. A table with # indexes can be used to originze the data... + +In regards to benchmarking. If you see my bench marks and are wondering they are 10x better its because I am using luajit for my tests. I highly recommend using luajit for my library, but lua 5.1 will work just as well, but not as fast. diff --git a/examples/lanesintegratetest7.lua b/examples/lanesintegratetest7.lua index 7a9f2df..4d8c3b4 100644 --- a/examples/lanesintegratetest7.lua +++ b/examples/lanesintegratetest7.lua @@ -1,6 +1,7 @@ -package.path="?/init.lua;"..package.path +-- Creating the object using lanes manager to show case this. Examples has the file for love2d local GLOBAL,sThread=require("multi.integration.lanesManager").init() -jQueue=multi:newSystemThreadedJobQueue() -- this internally creates System threads, We told it to use a maximum of 3 cores at any given time +jQueue=multi:newSystemThreadedJobQueue(n) -- this internally creates System threads. By defualt it will use the # of processors on your system You can set this number though. +-- Only create 1 jobqueue! For now making more than 1 is buggy. You only really need one though. Just register new functions if you want 1 queue to do more. The one reason though is keeping track of jobIDs. I have an idea that I will roll out in the next update. jQueue:registerJob("TEST_JOB",function(a,s) math.randomseed(s) -- We will push a random # @@ -8,19 +9,20 @@ jQueue:registerJob("TEST_JOB",function(a,s) return math.random(0,255) -- send the result to the main thread end) jQueue:registerJob("TEST_JOB2",function() - print("Test Works!") + print("Test Works!") -- this is called from the job since it is registered on the same queue end) -tableOfOrder={} -jQueue.OnJobCompleted(function(JOBID,n) +tableOfOrder={} -- This is how we will keep order of our completed jobs. There is no guarantee that the order will be correct +jQueue.OnJobCompleted(function(JOBID,n) -- whenever a job is completed you hook to the event that is called. This passes the JOBID folled by the returns of the job -- JOBID is the completed job, starts at 1 and counts up by 1. - -- Threads finish at different times so jobids may be returned out of order! Be sure to have a way to order them + -- Threads finish at different times so jobids may be passed out of order! Be sure to have a way to order them tableOfOrder[JOBID]=n -- we order ours by putting them into a table if #tableOfOrder==10 then print("We got all of the pieces!") end end) +-- LEts push the jobs now for i=1,10 do -- Job Name of registered function, ... varargs jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000)) end print("I pushed all of the jobs :)") -multi:mainloop() +multi:mainloop() -- Start the main loop :D diff --git a/examples/love2d Threading Example/main5.lua b/examples/love2d Threading Example/main5.lua index 3c50850..94a8a45 100644 --- a/examples/love2d Threading Example/main5.lua +++ b/examples/love2d Threading Example/main5.lua @@ -1,24 +1,3 @@ ---~ require("core.Library") ---~ GLOBAL,sThread=require("multi.integration.loveManager").init() -- load the love2d version of the lanesManager and requires the entire multi library ---~ require("core.GuiManager") ---~ gui.ff.Color=Color.Black ---~ test=multi:newSystemThreadedTable("YO"):init() ---~ test["test1"]="lol" ---~ multi:newSystemThread("test",function() ---~ tab=sThread.waitFor("YO"):init() ---~ print(tab["test1"]) ---~ sThread.sleep(3) ---~ tab["test2"]="Whats so funny?" ---~ end) ---~ multi:newThread("test2",function() ---~ thread.sleep(1) ---~ print(test:waitFor("test2")) ---~ t.text="DONE!" ---~ end) ---~ t=gui:newTextLabel("no done yet!",0,0,300,100) ---~ t:centerX() ---~ t:centerY() - require("core.Library") GLOBAL,sThread=require("multi.integration.loveManager").init() -- load the love2d version of the lanesManager and requires the entire multi library require("core.GuiManager") diff --git a/examples/love2d Threading Example/multi/init.lua b/examples/love2d Threading Example/multi/init.lua index cbd74b3..feadba6 100644 --- a/examples/love2d Threading Example/multi/init.lua +++ b/examples/love2d Threading Example/multi/init.lua @@ -45,8 +45,8 @@ function print(...) end end multi = {} -multi.Version="1.8.2" -multi._VERSION="1.8.2" +multi.Version="1.8.4" +multi._VERSION="1.8.4" multi.stage='stable' multi.__index = multi multi.Mainloop={} @@ -108,11 +108,13 @@ function multi:setThrestimed(n) end function multi:getLoad() return multi:newFunction(function(self) + multi.scheduler:Pause() local sample=#multi.Mainloop local FFloadtest=0 multi:benchMark(multi.threstimed):OnBench(function(_,l3) FFloadtest=l3*(1/multi.threstimed) end) self:hold(function() return FFloadtest~=0 end) local val=FFloadtest/sample + multi.scheduler:Resume() if val>multi.threshold then return 0 else @@ -617,7 +619,11 @@ function multi:hold(task) env:OnEvent(function(envt) envt:Pause() envt.Active=false end) while env.Active do if love then - self.Parent:lManager() + if love.graphics then + self.Parent:lManager() + else + self.Parent:Do_Order() + end else self.Parent:Do_Order() end @@ -978,7 +984,110 @@ function multi:mainloop() else return "Already Running!" end - --print("Did you call multi:Stop()? This method should not be used when using multi:mainloop() unless of course you wanted to stop it! you can restart the multi, by using multi:reboot() and calling multi:mainloop() again or by using multi:uManager()") +end +function multi:protectedMainloop() + multi:protect() + if not multi.isRunning then + multi.isRunning=true + for i=1,#self.Tasks do + self.Tasks[i](self) + end + rawset(self,'Start',self.clock()) + while self.Active do + self:Do_Order() + end + else + return "Already Running!" + end +end +function multi:unprotectedMainloop() + multi:unProtect() + if not multi.isRunning then + multi.isRunning=true + for i=1,#self.Tasks do + self.Tasks[i](self) + end + rawset(self,'Start',self.clock()) + while self.Active do + local Loop=self.Mainloop + _G.ID=0 + for _D=#Loop,1,-1 do + if Loop[_D] then + if Loop[_D].Active then + Loop[_D].Id=_D + self.CID=_D + Loop[_D]:Act() + end + end + end + end + else + return "Already Running!" + end +end +function multi:prioritizedMainloop1() + multi:enablePriority() + if not multi.isRunning then + multi.isRunning=true + for i=1,#self.Tasks do + self.Tasks[i](self) + end + rawset(self,'Start',self.clock()) + while self.Active do + local Loop=self.Mainloop + _G.ID=0 + local PS=self + for _D=#Loop,1,-1 do + if Loop[_D] then + if (PS.PList[PS.PStep])%Loop[_D].Priority==0 then + if Loop[_D].Active then + Loop[_D].Id=_D + self.CID=_D + Loop[_D]:Act() + end + end + end + end + PS.PStep=PS.PStep+1 + if PS.PStep>7 then + PS.PStep=1 + end + end + else + return "Already Running!" + end +end +function multi:prioritizedMainloop2() + multi:enablePriority2() + if not multi.isRunning then + multi.isRunning=true + for i=1,#self.Tasks do + self.Tasks[i](self) + end + rawset(self,'Start',self.clock()) + while self.Active do + local Loop=self.Mainloop + _G.ID=0 + local PS=self + for _D=#Loop,1,-1 do + if Loop[_D] then + if (PS.PStep)%Loop[_D].Priority==0 then + if Loop[_D].Active then + Loop[_D].Id=_D + self.CID=_D + Loop[_D]:Act() + end + end + end + end + PS.PStep=PS.PStep+1 + if PS.PStep>self.Priority_Idle then + PS.PStep=1 + end + end + else + return "Already Running!" + end end function multi._tFunc(self,dt) for i=1,#self.Tasks do @@ -1494,7 +1603,7 @@ function multi:newThread(name,func) end multi:setDomainName("Threads") multi:setDomainName("Globals") -multi.scheduler=multi:newUpdater() +multi.scheduler=multi:newLoop() multi.scheduler.Type="scheduler" function multi.scheduler:setStep(n) self.skip=tonumber(n) or 24 @@ -1503,7 +1612,7 @@ multi.scheduler.skip=0 multi.scheduler.counter=0 multi.scheduler.Threads=multi:linkDomain("Threads") multi.scheduler.Globals=multi:linkDomain("Globals") -multi.scheduler:OnUpdate(function(self) +multi.scheduler:OnLoop(function(self) self.counter=self.counter+1 for i=#self.Threads,1,-1 do ret={} @@ -1557,7 +1666,6 @@ multi.scheduler:OnUpdate(function(self) end end end) -multi.scheduler:setStep() multi.scheduler:Pause() multi.OnError=multi:newConnection() function multi:newThreadedAlarm(name,set) diff --git a/examples/love2d Threading Example/multi/integration/lanesManager.lua b/examples/love2d Threading Example/multi/integration/lanesManager.lua index 8d07bf1..9c0e60b 100644 --- a/examples/love2d Threading Example/multi/integration/lanesManager.lua +++ b/examples/love2d Threading Example/multi/integration/lanesManager.lua @@ -125,12 +125,9 @@ function multi:newSystemThread(name,func) end) return c end -print("Intergrated Lanes!") -multi.intergration={} -- for module creators -multi.intergration.GLOBAL=GLOBAL -multi.intergration.THREAD=THREAD -multi.intergration.lanes={} -multi.intergration.lanes.GLOBAL=GLOBAL -- for module creators -multi.intergration.lanes.THREAD=THREAD -- for module creators -require("multi.intergration.shared.shared") +print("Integrated Lanes!") +multi.integration={} -- for module creators +multi.integration.GLOBAL=GLOBAL +multi.integration.THREAD=THREAD +require("multi.integration.shared.shared") return {init=function() return GLOBAL,THREAD end} diff --git a/examples/love2d Threading Example/multi/integration/loveManager.lua b/examples/love2d Threading Example/multi/integration/loveManager.lua index 7a332ab..2440ade 100644 --- a/examples/love2d Threading Example/multi/integration/loveManager.lua +++ b/examples/love2d Threading Example/multi/integration/loveManager.lua @@ -2,14 +2,14 @@ require("multi.compat.love2d") function multi:canSystemThread() return true end -multi.intergration={} -multi.intergration.love2d={} -multi.intergration.love2d.ThreadBase=[[ +multi.integration={} +multi.integration.love2d={} +multi.integration.love2d.ThreadBase=[[ __THREADNAME__=({...})[1] require("love.filesystem") require("love.system") require("love.timer") -require("multi.all") +require("multi") GLOBAL={} setmetatable(GLOBAL,{ __index=function(t,k) @@ -29,6 +29,7 @@ setmetatable(GLOBAL,{ function __sync__() local data=__mythread__:pop() while data do + love.timer.sleep(.001) if type(data)=="string" then local cmd,tp,name,d=data:match("(%S-) (%S-) (%S-) (.+)") if name=="__DIEPLZ"..__THREADNAME__.."__" then @@ -79,7 +80,7 @@ function resolveType(tp,d) elseif tp=="bool" then return (d=="true") elseif tp=="function" then - return loadDump(d) + return loadDump("[==["..d.."]==]") elseif tp=="table" then return loadstring("return "..d)() elseif tp=="nil" then @@ -91,11 +92,11 @@ end function resolveData(v) local data="" if type(v)=="table" then - data=ToStr(v) + return ToStr(v) elseif type(v)=="function" then - data=dump(v) + return dump(v) elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then - data=tostring(v) + return tostring(v) end return data end @@ -152,24 +153,16 @@ end function sThread.hold(n) repeat __sync__() until n() end -updater=multi:newUpdater() -updater:OnUpdate(function(self) - local data=__mythread__:pop() - while data do - if type(data)=="string" then - local cmd,tp,name,d=data:match("(%S-) (%S-) (%S-) (.+)") - if name=="__DIEPLZ"..__THREADNAME__.."__" then - error("Thread: "..__THREADNAME__.." has been stopped!") - end - if cmd=="SYNC" then - __proxy__[name]=resolveType(tp,d) - end - else - __proxy__[name]=data - end - data=__mythread__:pop() +multi:newLoop(function(self) + self:Pause() + local ld=multi:getLoad() + self:Resume() + if ld<80 then + love.timer.sleep(.01) end end) +updater=multi:newUpdater() +updater:OnUpdate(__sync__) func=loadDump([=[INSERT_USER_CODE]=])() multi:mainloop() ]] @@ -191,7 +184,7 @@ setmetatable(GLOBAL,{ end, }) THREAD={} -- Allow main thread to interact with these objects as well -multi.intergration.love2d.mainChannel=love.thread.getChannel("__MainChan__") +multi.integration.love2d.mainChannel=love.thread.getChannel("__MainChan__") function ToStr(val, name, skipnewlines, depth) skipnewlines = skipnewlines or false depth = depth or 0 @@ -228,7 +221,7 @@ function resolveType(tp,d) elseif tp=="bool" then return (d=="true") elseif tp=="function" then - return loadDump(d) + return loadDump("[==["..d.."]==]") elseif tp=="table" then return loadstring("return "..d)() elseif tp=="nil" then @@ -240,11 +233,11 @@ end function resolveData(v) local data="" if type(v)=="table" then - data=ToStr(v) + return ToStr(v) elseif type(v)=="function" then - data=dump(v) + return dump(v) elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then - data=tostring(v) + return tostring(v) end return data end @@ -274,10 +267,10 @@ function multi:newSystemThread(name,func) -- the main method local c={} c.name=name c.ID=c.name.."" - c.thread=love.thread.newThread(multi.intergration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func))) + c.thread=love.thread.newThread(multi.integration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func))) c.thread:start(c.ID) function c:kill() - multi.intergration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__" + multi.integration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__" end return c end @@ -310,11 +303,11 @@ function THREAD.hold(n) multi.OBJ_REF:Resume() end __channels__={} -multi.intergration.GLOBAL=GLOBAL -multi.intergration.THREAD=THREAD +multi.integration.GLOBAL=GLOBAL +multi.integration.THREAD=THREAD updater=multi:newUpdater() updater:OnUpdate(function(self) - local data=multi.intergration.love2d.mainChannel:pop() + local data=multi.integration.love2d.mainChannel:pop() while data do --print("MAIN:",data) if type(data)=="string" then @@ -342,19 +335,21 @@ updater:OnUpdate(function(self) else __proxy__[name]=data end - data=multi.intergration.love2d.mainChannel:pop() + data=multi.integration.love2d.mainChannel:pop() end end) -require("multi.intergration.shared.shared") -print("Intergrated Love2d!") +require("multi.integration.shared.shared") +print("Integrated Love2d!") return { init=function(t) if t then if t.threadNamespace then - multi.intergration.love2d.ThreadBase:gsub("sThread",t.threadNamespace) + multi.integration.THREADNAME=t.threadNamespace + multi.integration.love2d.ThreadBase:gsub("sThread",t.threadNamespace) end if t.globalNamespace then - multi.intergration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace) + multi.integration.GLOBALNAME=t.globalNamespace + multi.integration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace) end end return GLOBAL,THREAD diff --git a/examples/love2d Threading Example/multi/integration/shared/shared.lua b/examples/love2d Threading Example/multi/integration/shared/shared.lua index 06bb8cf..92a1b44 100644 --- a/examples/love2d Threading Example/multi/integration/shared/shared.lua +++ b/examples/love2d Threading Example/multi/integration/shared/shared.lua @@ -280,7 +280,6 @@ if love then GLOBAL["THREADED_JQ"]=nil -- remove it GLOBAL["THREADED_JQO"]=nil -- remove it multi:newLoop(function() - sThread.sleep(.001) -- lets allow cpu time for other processes on our system! local job=JQI:pop() if job then local ID=table.remove(job,1) -- return and remove diff --git a/multi/init.lua b/multi/init.lua index 42f8b74..feadba6 100644 --- a/multi/init.lua +++ b/multi/init.lua @@ -108,11 +108,13 @@ function multi:setThrestimed(n) end function multi:getLoad() return multi:newFunction(function(self) + multi.scheduler:Pause() local sample=#multi.Mainloop local FFloadtest=0 multi:benchMark(multi.threstimed):OnBench(function(_,l3) FFloadtest=l3*(1/multi.threstimed) end) self:hold(function() return FFloadtest~=0 end) local val=FFloadtest/sample + multi.scheduler:Resume() if val>multi.threshold then return 0 else @@ -617,7 +619,11 @@ function multi:hold(task) env:OnEvent(function(envt) envt:Pause() envt.Active=false end) while env.Active do if love then - self.Parent:lManager() + if love.graphics then + self.Parent:lManager() + else + self.Parent:Do_Order() + end else self.Parent:Do_Order() end diff --git a/multi/integration/loveManager.lua b/multi/integration/loveManager.lua index 0a144d9..2440ade 100644 --- a/multi/integration/loveManager.lua +++ b/multi/integration/loveManager.lua @@ -153,6 +153,14 @@ end function sThread.hold(n) repeat __sync__() until n() end +multi:newLoop(function(self) + self:Pause() + local ld=multi:getLoad() + self:Resume() + if ld<80 then + love.timer.sleep(.01) + end +end) updater=multi:newUpdater() updater:OnUpdate(__sync__) func=loadDump([=[INSERT_USER_CODE]=])() diff --git a/multi/integration/shared/shared.lua b/multi/integration/shared/shared.lua index 06bb8cf..92a1b44 100644 --- a/multi/integration/shared/shared.lua +++ b/multi/integration/shared/shared.lua @@ -280,7 +280,6 @@ if love then GLOBAL["THREADED_JQ"]=nil -- remove it GLOBAL["THREADED_JQO"]=nil -- remove it multi:newLoop(function() - sThread.sleep(.001) -- lets allow cpu time for other processes on our system! local job=JQI:pop() if job then local ID=table.remove(job,1) -- return and remove -- 2.43.0