Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme?
My multitasking library for lua
To install copy the multi folder into your enviroment and you are good to go
It is a pure lua binding if you ingore the integrations and the love2d compat
If you find any bugs or have any issues please let me know :)
Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme? Also I added a Testing Branch. That Branch will have the latest updates, but those updates may be unstable. I like to keep the master as bug free as possible
In Changes you’ll find documentation for(In Order):
My multitasking library for lua
To install copy the multi folder into your enviroment and you are good to go
It is a pure lua binding if you ingore the integrations and the love2d compat
If you find any bugs or have any issues please let me know :)
If you don’t see a table of contents try using the ReadMe.html file. It is eaiser to navigate the readme
luarocks install multi
For real-time assistance with my libraries! A place where you can ask questions and get help with any of my libraries. Also you can request features and stuff there as well.
https://discord.gg/U8UspuA
In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P
Add system threads for love2d that works like the lanesManager (loveManager, slight differences). Improve performance of the library Improve coroutine based threading scheduling Improve love2d Idle thread cpu usage… Tricky Look at the rambling section for insight. Add more features to support module creators Make a framework for eaiser thread task distributing Fix Error handling on threaded multi objects Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance! Add multi:OnError(function(obj,err)) sThread.wrap(obj) May or may not be completed Theory: Allows interaction in one thread to affect it in another. The addition to threaded tables may make this possible! SystemThreaded Actors — After some tests i figured out a way to make this work… It will work slightly different though. This is due to the actor needing to be splittable… LoadBalancing for system threads (Once SystemThreaded Actors are done) Add more integrations Finish the wiki stuff. (11% done) Test for unknown bugs Known Bugs/Issues
In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P
Another bug concerns the SystemThreadedJobQueue, Only 1 can be used for now… Vreating more may not be a good idea.
Usage:
function<
end
end)
multi:mainloop()
-
Output (Change the value inc as indicated in the comment to see the outcomes!)
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Loop timed out! tloop Trying again…
Looping…
Looping…
Looping…
Looping…
Looping…
We did it! 1 2 3
Changes
Updated from 1.8.2 to 1.8.3
Added:
New Mainloop functions Below you can see the slight differences… Function overhead is not too bad in lua, but has a real difference. multi:mainloop() and multi:unprotectedMainloop() use the same algorithm yet the dedicated unprotected one is slightly faster due to having less function overhead.
-- multi:mainloop()* — Bench: 16830003 Steps in 3 second(s)!
- multi:protectedMainloop() — Bench: 16699308 Steps in 3 second(s)!
- multi:unprotectedMainloop() — Bench: 16976627 Steps in 3 second(s)!
- multi:prioritizedMainloop1() — Bench: 15007133 Steps in 3 second(s)!
- multi:prioritizedMainloop2() — Bench: 15526248 Steps in 3 second(s)!
* The OG mainloop function remains the same and old methods to achieve what we have with the new ones still exist
These new methods help by removing function overhead that is caused through the original mainloop function. The one downside is that you no longer have the flexiblity to change the processing during runtime.
However there is a work around! You can use processes to run multiobjs as well and use the other methods on them.
I may make a full comparison between each method and which is faster, but for now trust that the dedicated ones with less function overhead are infact faster. Not by much but still faster. :D
Updated from 1.8.1 to 1.8.2
Added:
+
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Looping…
Loop timed out! tloop Trying again…
Looping…
Looping…
Looping…
Looping…
Looping…
We did it! 1 2 3
Added:
First you need to create the object
This works the same way as love2d as it does with lanes… It is getting increasing harder to make both work the same way with speed in mind… Anyway…
-- Creating the object using lanes manager to show case this. Examples has the file for love2d
+local GLOBAL,sThread=require("multi.integration.lanesManager").init()
+jQueue=multi:newSystemThreadedJobQueue(n) -- this internally creates System threads. By defualt it will use the # of processors on your system You can set this number though.
+-- Only create 1 jobqueue! For now making more than 1 is buggy. You only really need one though. Just register new functions if you want 1 queue to do more. The one reason though is keeping track of jobIDs. I have an idea that I will roll out in the next update.
+jQueue:registerJob("TEST_JOB",function(a,s)
+ math.randomseed(s)
+ -- We will push a random #
+ TEST_JOB2() -- You can call other registered functions as well!
+ return math.random(0,255) -- send the result to the main thread
+end)
+jQueue:registerJob("TEST_JOB2",function()
+ print("Test Works!") -- this is called from the job since it is registered on the same queue
+end)
+tableOfOrder={} -- This is how we will keep order of our completed jobs. There is no guarantee that the order will be correct
+jQueue.OnJobCompleted(function(JOBID,n) -- whenever a job is completed you hook to the event that is called. This passes the JOBID folled by the returns of the job
+ -- JOBID is the completed job, starts at 1 and counts up by 1.
+ -- Threads finish at different times so jobids may be passed out of order! Be sure to have a way to order them
+ tableOfOrder[JOBID]=n -- we order ours by putting them into a table
+ if #tableOfOrder==10 then
+ print("We got all of the pieces!")
+ end
+end)
+-- Lets push the jobs now
+for i=1,10 do -- Job Name of registered function, ... varargs
+ jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000))
+end
+print("I pushed all of the jobs :)")
+multi:mainloop() -- Start the main loop :D
+Thats it from this version!
Added:
New Mainloop functions Below you can see the slight differences… Function overhead is not too bad in lua, but has a real difference. multi:mainloop() and multi:unprotectedMainloop() use the same algorithm yet the dedicated unprotected one is slightly faster due to having less function overhead.
* The OG mainloop function remains the same and old methods to achieve what we have with the new ones still exist
These new methods help by removing function overhead that is caused through the original mainloop function. The one downside is that you no longer have the flexiblity to change the processing during runtime.
However there is a work around! You can use processes to run multiobjs as well and use the other methods on them.
I may make a full comparison between each method and which is faster, but for now trust that the dedicated ones with less function overhead are infact faster. Not by much but still faster. :D
Added:
The threaded table is setup just like the threaded queue.
It provids GLOBAL like features without having to write to GLOBAL!
This is useful for module creators who want to keep their data private, but also use GLOBAL like coding.
It has a few features that makes it a bit better than plain ol GLOBAL (For now…)
(ThreadedTable - TT for short)
we also have the “sync” method, this one was made for love2d because we do a syncing trick to get data in a table format. The lanes side has a sync method as well so no worries. Using indexing calls sync once and may grab your variable. This allows you to have the lanes indexing ‘like’ syntax when doing regular indexing in love2d side of the module. As of right now both sides work flawlessly! And this effect is now the GLOBAL as well
On GLOBALS sync is a internal method for keeping the GLOBAL table in order. You can still use sThread.waitFor(name) to wait for variables that may of may not yet exist!
Time for some examples:
"test2","no done yet!",0,0,300,100)
t:centerX()
t:centerY()
-Updated from 1.8.0 to 1.8.1
No real change!
Changed the structure of the library. Combined the coroutine based threads into the core!
Only compat and integrations are not part of the core and never will be by nature.
This should make the library more convient to use.
I left multi/all.lua file so if anyone had libraries/projects that used that it will still work!
Updated from 1.7.6 to 1.8.0
(How much thread could a thread thread if a thread could thread thread?)
Added:
No real change!
Changed the structure of the library. Combined the coroutine based threads into the core!
Only compat and integrations are not part of the core and never will be by nature.
This should make the library more convient to use.
I left multi/all.lua file so if anyone had libraries/projects that used that it will still work!
Updated from 1.7.6 to 1.8.0
(How much thread could a thread thread if a thread could thread thread?)
Added:
"test!",end
end)
multi:mainloop()
-Updated from 1.7.5 to 1.7.6
Fixed:
Typos like always
Added:
multi:getPlatform() — returns “love2d” if using the love2d platform or returns “lanes” if using lanes for threading
examples files
In Events added method setTask(func)
The old way still works and is more convient to be honest, but I felt a method to do this was ok.
Updated:
some example files to reflect changes to the core. Changes allow for less typing
loveManager to require the compat if used so you don’t need 2 require line to retrieve the library
Updated from 1.7.4 to 1.7.5
Fixed some typos in the readme… (I am sure there are more there are always more)
Added more features for module support
TODO:
Work on performance of the library… I see 3 places where I can make this thing run quicker
I’ll show case some old versions of the multitasking library eventually so you can see its changes in days past!
Updated from 1.7.3 to 1.7.4
Added: the example folder which will be populated with more examples in the near future!
The loveManager integration that mimics the lanesManager integration almost exactly to keep coding in both enviroments as close to possible. This is done mostly for library creation support!
An example of the loveManager in action using almost the same code as the lanesintergreationtest2.lua
NOTE: This code has only been tested to work on love2d version 1.10.2 thoough it should work version 0.9.0
Updated from 1.7.5 to 1.7.6Fixed:
Typos like always
Added:
multi:getPlatform() — returns “love2d” if using the love2d platform or returns “lanes” if using lanes for threading
examples files
In Events added method setTask(func)
The old way still works and is more convient to be honest, but I felt a method to do this was ok.
Updated:
some example files to reflect changes to the core. Changes allow for less typing
loveManager to require the compat if used so you don’t need 2 require line to retrieve the library
Updated from 1.7.4 to 1.7.5
Fixed some typos in the readme… (I am sure there are more there are always more)
Added more features for module support
TODO:
Work on performance of the library… I see 3 places where I can make this thing run quicker
I’ll show case some old versions of the multitasking library eventually so you can see its changes in days past!
Updated from 1.7.3 to 1.7.4
Added: the example folder which will be populated with more examples in the near future!
The loveManager integration that mimics the lanesManager integration almost exactly to keep coding in both enviroments as close to possible. This is done mostly for library creation support!
An example of the loveManager in action using almost the same code as the lanesintergreationtest2.lua
NOTE: This code has only been tested to work on love2d version 1.10.2 thoough it should work version 0.9.0
"Bench"]=3"no done yet!",0,0,300,100)
t:centerX()
t:centerY()
-
Updated from 1.7.2 to 1.7.3
Changed how requiring the library works!
require("multi.all") Will still work as expected; however, with the exception of threading, compat, and integrations everything else has been moved into the core of the library.
Updated from 1.7.2 to 1.7.3Changed how requiring the library works!
require("multi.all") Will still work as expected; however, with the exception of threading, compat, and integrations everything else has been moved into the core of the library.
require("multi.step")
require("multi.task")
-- ^ they are all part of the core now
-
Updated from 1.7.1 to 1.7.2
Moved updaters, loops, and alarms into the init.lua file. I consider them core features and they are referenced in the init.lua file so they need to exist there. Threaded versions are still separate though. Added another example file
Updated from 1.7.0 to 1.7.1 Bug fixes only
Updated from 1.6.0 to 1.7.0
Modified: multi.integration.lanesManager.lua
It is now in a stable and simple state Works with the latest lanes version! Tested with version 3.11 I cannot promise that everything will work with eariler versions. Future versions are good though.
Example Usage:
sThread is a handle to a global interface for system threads to interact with themself
thread is the interface for multithreads as seen in the threading section
GLOBAL a table that can be used throughout each and every thread
sThreads have a few methods
sThread.set(name,val) — you can use the GLOBAL table instead modifies the same table anyway
sThread.get(name) — you can use the GLOBAL table instead modifies the same table anyway
sThread.waitFor(name) — waits until a value exists, if it does it returns it
sThread.getCores() — returns the number of cores on your cpu
sThread.sleep(n) — sleeps for a bit stopping the entire thread from running
sThread.hold(n) — sleeps until a condition is met
Updated from 1.7.1 to 1.7.2Moved updaters, loops, and alarms into the init.lua file. I consider them core features and they are referenced in the init.lua file so they need to exist there. Threaded versions are still separate though. Added another example file
Updated from 1.7.0 to 1.7.1 Bug fixes only
Updated from 1.6.0 to 1.7.0
Modified: multi.integration.lanesManager.lua
It is now in a stable and simple state Works with the latest lanes version! Tested with version 3.11 I cannot promise that everything will work with eariler versions. Future versions are good though.
Example Usage:
sThread is a handle to a global interface for system threads to interact with themself
thread is the interface for multithreads as seen in the threading section
GLOBAL a table that can be used throughout each and every thread
sThreads have a few methods
sThread.set(name,val) — you can use the GLOBAL table instead modifies the same table anyway
sThread.get(name) — you can use the GLOBAL table instead modifies the same table anyway
sThread.waitFor(name) — waits until a value exists, if it does it returns it
sThread.getCores() — returns the number of cores on your cpu
sThread.sleep(n) — sleeps for a bit stopping the entire thread from running
sThread.hold(n) — sleeps until a condition is met
"test",end)
multi:mainloop()
-
Updated from 1.5.0 to 1.6.0
Changed: steps and loops
Updated from 1.5.0 to 1.6.0Changed: steps and loops
function
">require("multi.all")
require("multi.compat.backwards[1,5,0]") -- allows for the use of features that were scrapped/changed in 1.6.0+
-
Updated from 1.4.1 to 1.5.0
Added:
-- An easy way to manage timeouts
- Small bug fixes
1.4.1 - First Public release of the library
IMPORTANT:
Every update I make aims to make things simpler more efficent and just better, but a lot of old code, which can be really big, uses a lot of older features. I know the pain of having to rewrite everything. My promise to my library users is that I will always have backwards support for older features! New ways may exist that are quicker and eaiser, but the old features/methods will be supported.
+
Updated from 1.4.1 to 1.5.0
Added:
+- An easy way to manage timeouts
- Small bug fixes
1.4.1 - First Public release of the library
IMPORTANT:
Every update I make aims to make things simpler more efficent and just better, but a lot of old code, which can be really big, uses a lot of older features. I know the pain of having to rewrite everything. My promise to my library users is that I will always have backwards support for older features! New ways may exist that are quicker and eaiser, but the old features/methods will be supported.
Rambling
Love2d: Sleeping reduces the cpu time making my load detection think the system is under more load, thus preventing it from sleeping… I will look into other means. As of right now it will not eat all of your cpu if threads are active. For now I suggest killing threads that aren’t needed anymore. On lanes threads at idle use 0% cpu and it is amazing. A state machine may solve what I need though. One state being idle state that sleeps and only goes into the active state if a job request or data is sent to it… after some time of not being under load it wil switch back into the idle state… We’ll see what happens.
Love2d doesn’t like to send functions through channels. By defualt it does not support this. I achieve this by dumping the function and loadstring it on the thread. This however is slow. For the System Threaded Job Queue I had to change my original idea of sending functions as jobs. The current way you do it now is register a job functions once and then call that job across the thread through a queue. Each worker thread pops from the queue and returns the job. The Job ID is automatically updated and allows you to keep track of the order that the data comes in. A table with # indexes can be used to originze the data…
In regards to benchmarking. If you see my bench marks and are wondering they are 10x better its because I am using luajit for my tests. I highly recommend using luajit for my library, but lua 5.1 will work just as well, but not as fast.
diff --git a/README.md b/README.md
index 7d133a2..f231ca5 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,15 @@
# multi Version: 1.8.4 (System Threaded Job Queues)
**Note: The changes section has information on how to use the new features as they come out. Why put the infomation twice on the readme?** Also I added a Testing Branch. That Branch will have the latest updates, but those updates may be unstable. I like to keep the master as bug free as possible
+In Changes you'll find documentation for(In Order):
+- System Threaded Job Queues
+- New mainloop functions
+- System Threaded Tables
+- System Threaded Benchmark
+- System Threaded Queues
+- Threading related features
+- And backwards compat stuff
+
My multitasking library for lua
To install copy the multi folder into your enviroment and you are good to go
@@ -8,6 +17,8 @@ It is a pure lua binding if you ingore the integrations and the love2d compat
+
[TOC]
INSTALLING
@@ -24,7 +35,8 @@ Planned features/TODO
---------------------
- [x] ~~Add system threads for love2d that works like the lanesManager (loveManager, slight differences).~~
- [x] ~~Improve performance of the library~~
-- [ ] Improve coroutine based threading scheduling
+- [x] ~~Improve coroutine based threading scheduling~~
+- [ ] Improve love2d Idle thread cpu usage... Tricky Look at the rambling section for insight.
- [x] ~~Add more features to support module creators~~
- [x] ~~Make a framework for eaiser thread task distributing~~
- [x] ~~Fix Error handling on threaded multi objects~~ Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance!
@@ -40,6 +52,8 @@ Known Bugs/Issues
-----------------
In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to...) not use the multi library within the thread. A fix for this is to call `multi:Stop()` when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data... You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P
+Another bug concerns the SystemThreadedJobQueue, Only 1 can be used for now... Vreating more may not be a good idea.
+
Usage:
-----
```lua
@@ -785,7 +799,53 @@ We did it! 1 2 3
Changes
-------
-Updated from 1.8.2 to 1.8.3
+Updated from 1.8.3 to 1.8.4
+---------------------------
+Added:
+- multi:newSystemThreadedJobQueue()
+- Improved stability of the library
+- Fixed a bug that made the benchmark and getload commands non-thread(coroutine) safe
+- Tweaked the loveManager to help improve idle cpu usage
+- Minor tweaks to the coroutine scheduling
+
+# Using multi:newSystemThreadedJobQueue()
+First you need to create the object
+This works the same way as love2d as it does with lanes... It is getting increasing harder to make both work the same way with speed in mind... Anyway...
+```lua
+-- Creating the object using lanes manager to show case this. Examples has the file for love2d
+local GLOBAL,sThread=require("multi.integration.lanesManager").init()
+jQueue=multi:newSystemThreadedJobQueue(n) -- this internally creates System threads. By defualt it will use the # of processors on your system You can set this number though.
+-- Only create 1 jobqueue! For now making more than 1 is buggy. You only really need one though. Just register new functions if you want 1 queue to do more. The one reason though is keeping track of jobIDs. I have an idea that I will roll out in the next update.
+jQueue:registerJob("TEST_JOB",function(a,s)
+ math.randomseed(s)
+ -- We will push a random #
+ TEST_JOB2() -- You can call other registered functions as well!
+ return math.random(0,255) -- send the result to the main thread
+end)
+jQueue:registerJob("TEST_JOB2",function()
+ print("Test Works!") -- this is called from the job since it is registered on the same queue
+end)
+tableOfOrder={} -- This is how we will keep order of our completed jobs. There is no guarantee that the order will be correct
+jQueue.OnJobCompleted(function(JOBID,n) -- whenever a job is completed you hook to the event that is called. This passes the JOBID folled by the returns of the job
+ -- JOBID is the completed job, starts at 1 and counts up by 1.
+ -- Threads finish at different times so jobids may be passed out of order! Be sure to have a way to order them
+ tableOfOrder[JOBID]=n -- we order ours by putting them into a table
+ if #tableOfOrder==10 then
+ print("We got all of the pieces!")
+ end
+end)
+-- Lets push the jobs now
+for i=1,10 do -- Job Name of registered function, ... varargs
+ jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000))
+end
+print("I pushed all of the jobs :)")
+multi:mainloop() -- Start the main loop :D
+```
+
+Thats it from this version!
+
+Updated from 1.8.2 to 1.8.3
+---------------------------
Added:
**New Mainloop functions** Below you can see the slight differences... Function overhead is not too bad in lua, but has a real difference. multi:mainloop() and multi:unprotectedMainloop() use the same algorithm yet the dedicated unprotected one is slightly faster due to having less function overhead.
- multi:mainloop()\* -- Bench: 16830003 Steps in 3 second(s)!
@@ -801,7 +861,9 @@ These new methods help by removing function overhead that is caused through the
However there is a work around! You can use processes to run multiobjs as well and use the other methods on them.
I may make a full comparison between each method and which is faster, but for now trust that the dedicated ones with less function overhead are infact faster. Not by much but still faster. :D
-Updated from 1.8.1 to 1.8.2
+
+Updated from 1.8.1 to 1.8.2
+---------------------------
Added:
- multi:newsystemThreadedTable(name) NOTE: Metatables are not supported in transfers. However there is a work around obj:init() that you see does this. Take a look in the multi/integration/shared/shared.lua files to see how I did it!
- Modified the GLOBAL metatable to sync before doing its tests
@@ -864,7 +926,8 @@ t:centerX()
t:centerY()
```
-Updated from 1.8.0 to 1.8.1
+Updated from 1.8.0 to 1.8.1
+---------------------------
No real change!
Changed the structure of the library. Combined the coroutine based threads into the core!
Only compat and integrations are not part of the core and never will be by nature.
@@ -991,7 +1054,8 @@ multi:newThread("test!",function() -- this is a lua thread
end)
multi:mainloop()
```
-Updated from 1.7.5 to 1.7.6
+Updated from 1.7.5 to 1.7.6
+---------------------------
Fixed:
Typos like always
Added:
@@ -1003,7 +1067,8 @@ The old way still works and is more convient to be honest, but I felt a method t
Updated:
some example files to reflect changes to the core. Changes allow for less typing
loveManager to require the compat if used so you don't need 2 require line to retrieve the library
-Updated from 1.7.4 to 1.7.5
+Updated from 1.7.4 to 1.7.5
+---------------------------
Fixed some typos in the readme... (I am sure there are more there are always more)
Added more features for module support
TODO:
@@ -1011,7 +1076,8 @@ Work on performance of the library... I see 3 places where I can make this thing
I'll show case some old versions of the multitasking library eventually so you can see its changes in days past!
-Updated from 1.7.3 to 1.7.4
+Updated from 1.7.3 to 1.7.4
+---------------------------
Added: the example folder which will be populated with more examples in the near future!
The loveManager integration that mimics the lanesManager integration almost exactly to keep coding in both enviroments as close to possible. This is done mostly for library creation support!
An example of the loveManager in action using almost the same code as the lanesintergreationtest2.lua
@@ -1088,7 +1154,8 @@ t=gui:newTextLabel("no done yet!",0,0,300,100)
t:centerX()
t:centerY()
```
-Updated from 1.7.2 to 1.7.3
+Updated from 1.7.2 to 1.7.3
+---------------------------
Changed how requiring the library works!
`require("multi.all")` Will still work as expected; however, with the exception of threading, compat, and integrations everything else has been moved into the core of the library.
```lua
@@ -1104,12 +1171,15 @@ require("multi.task")
-- ^ they are all part of the core now
```
-Updated from 1.7.1 to 1.7.2
+Updated from 1.7.1 to 1.7.2
+---------------------------
Moved updaters, loops, and alarms into the init.lua file. I consider them core features and they are referenced in the init.lua file so they need to exist there. Threaded versions are still separate though. Added another example file
Updated from 1.7.0 to 1.7.1 Bug fixes only
+---------------------------
-Updated from 1.6.0 to 1.7.0
+Updated from 1.6.0 to 1.7.0
+---------------------------
Modified: multi.integration.lanesManager.lua
It is now in a stable and simple state Works with the latest lanes version! Tested with version 3.11 I cannot promise that everything will work with eariler versions. Future versions are good though.
Example Usage:
@@ -1162,7 +1232,8 @@ end)
multi:mainloop()
```
-Updated from 1.5.0 to 1.6.0
+Updated from 1.5.0 to 1.6.0
+---------------------------
Changed: steps and loops
```lua
-- Was
@@ -1186,11 +1257,20 @@ require("multi.all")
require("multi.compat.backwards[1,5,0]") -- allows for the use of features that were scrapped/changed in 1.6.0+
```
Updated from 1.4.1 to 1.5.0
+---------------------------
Added:
- An easy way to manage timeouts
- Small bug fixes
1.4.1 - First Public release of the library
+---------------------------
IMPORTANT:
Every update I make aims to make things simpler more efficent and just better, but a lot of old code, which can be really big, uses a lot of older features. I know the pain of having to rewrite everything. My promise to my library users is that I will always have backwards support for older features! New ways may exist that are quicker and eaiser, but the old features/methods will be supported.
+Rambling
+--------
+Love2d: Sleeping reduces the cpu time making my load detection think the system is under more load, thus preventing it from sleeping... I will look into other means. As of right now it will not eat all of your cpu if threads are active. For now I suggest killing threads that aren't needed anymore. On lanes threads at idle use 0% cpu and it is amazing. A state machine may solve what I need though. One state being idle state that sleeps and only goes into the active state if a job request or data is sent to it... after some time of not being under load it wil switch back into the idle state... We'll see what happens.
+
+Love2d doesn't like to send functions through channels. By defualt it does not support this. I achieve this by dumping the function and loadstring it on the thread. This however is slow. For the System Threaded Job Queue I had to change my original idea of sending functions as jobs. The current way you do it now is register a job functions once and then call that job across the thread through a queue. Each worker thread pops from the queue and returns the job. The Job ID is automatically updated and allows you to keep track of the order that the data comes in. A table with # indexes can be used to originze the data...
+
+In regards to benchmarking. If you see my bench marks and are wondering they are 10x better its because I am using luajit for my tests. I highly recommend using luajit for my library, but lua 5.1 will work just as well, but not as fast.
diff --git a/examples/lanesintegratetest7.lua b/examples/lanesintegratetest7.lua
index 7a9f2df..4d8c3b4 100644
--- a/examples/lanesintegratetest7.lua
+++ b/examples/lanesintegratetest7.lua
@@ -1,6 +1,7 @@
-package.path="?/init.lua;"..package.path
+-- Creating the object using lanes manager to show case this. Examples has the file for love2d
local GLOBAL,sThread=require("multi.integration.lanesManager").init()
-jQueue=multi:newSystemThreadedJobQueue() -- this internally creates System threads, We told it to use a maximum of 3 cores at any given time
+jQueue=multi:newSystemThreadedJobQueue(n) -- this internally creates System threads. By defualt it will use the # of processors on your system You can set this number though.
+-- Only create 1 jobqueue! For now making more than 1 is buggy. You only really need one though. Just register new functions if you want 1 queue to do more. The one reason though is keeping track of jobIDs. I have an idea that I will roll out in the next update.
jQueue:registerJob("TEST_JOB",function(a,s)
math.randomseed(s)
-- We will push a random #
@@ -8,19 +9,20 @@ jQueue:registerJob("TEST_JOB",function(a,s)
return math.random(0,255) -- send the result to the main thread
end)
jQueue:registerJob("TEST_JOB2",function()
- print("Test Works!")
+ print("Test Works!") -- this is called from the job since it is registered on the same queue
end)
-tableOfOrder={}
-jQueue.OnJobCompleted(function(JOBID,n)
+tableOfOrder={} -- This is how we will keep order of our completed jobs. There is no guarantee that the order will be correct
+jQueue.OnJobCompleted(function(JOBID,n) -- whenever a job is completed you hook to the event that is called. This passes the JOBID folled by the returns of the job
-- JOBID is the completed job, starts at 1 and counts up by 1.
- -- Threads finish at different times so jobids may be returned out of order! Be sure to have a way to order them
+ -- Threads finish at different times so jobids may be passed out of order! Be sure to have a way to order them
tableOfOrder[JOBID]=n -- we order ours by putting them into a table
if #tableOfOrder==10 then
print("We got all of the pieces!")
end
end)
+-- LEts push the jobs now
for i=1,10 do -- Job Name of registered function, ... varargs
jQueue:pushJob("TEST_JOB","This is a test!",math.random(1,1000000))
end
print("I pushed all of the jobs :)")
-multi:mainloop()
+multi:mainloop() -- Start the main loop :D
diff --git a/examples/love2d Threading Example/main5.lua b/examples/love2d Threading Example/main5.lua
index 3c50850..94a8a45 100644
--- a/examples/love2d Threading Example/main5.lua
+++ b/examples/love2d Threading Example/main5.lua
@@ -1,24 +1,3 @@
---~ require("core.Library")
---~ GLOBAL,sThread=require("multi.integration.loveManager").init() -- load the love2d version of the lanesManager and requires the entire multi library
---~ require("core.GuiManager")
---~ gui.ff.Color=Color.Black
---~ test=multi:newSystemThreadedTable("YO"):init()
---~ test["test1"]="lol"
---~ multi:newSystemThread("test",function()
---~ tab=sThread.waitFor("YO"):init()
---~ print(tab["test1"])
---~ sThread.sleep(3)
---~ tab["test2"]="Whats so funny?"
---~ end)
---~ multi:newThread("test2",function()
---~ thread.sleep(1)
---~ print(test:waitFor("test2"))
---~ t.text="DONE!"
---~ end)
---~ t=gui:newTextLabel("no done yet!",0,0,300,100)
---~ t:centerX()
---~ t:centerY()
-
require("core.Library")
GLOBAL,sThread=require("multi.integration.loveManager").init() -- load the love2d version of the lanesManager and requires the entire multi library
require("core.GuiManager")
diff --git a/examples/love2d Threading Example/multi/init.lua b/examples/love2d Threading Example/multi/init.lua
index cbd74b3..feadba6 100644
--- a/examples/love2d Threading Example/multi/init.lua
+++ b/examples/love2d Threading Example/multi/init.lua
@@ -45,8 +45,8 @@ function print(...)
end
end
multi = {}
-multi.Version="1.8.2"
-multi._VERSION="1.8.2"
+multi.Version="1.8.4"
+multi._VERSION="1.8.4"
multi.stage='stable'
multi.__index = multi
multi.Mainloop={}
@@ -108,11 +108,13 @@ function multi:setThrestimed(n)
end
function multi:getLoad()
return multi:newFunction(function(self)
+ multi.scheduler:Pause()
local sample=#multi.Mainloop
local FFloadtest=0
multi:benchMark(multi.threstimed):OnBench(function(_,l3) FFloadtest=l3*(1/multi.threstimed) end)
self:hold(function() return FFloadtest~=0 end)
local val=FFloadtest/sample
+ multi.scheduler:Resume()
if val>multi.threshold then
return 0
else
@@ -617,7 +619,11 @@ function multi:hold(task)
env:OnEvent(function(envt) envt:Pause() envt.Active=false end)
while env.Active do
if love then
- self.Parent:lManager()
+ if love.graphics then
+ self.Parent:lManager()
+ else
+ self.Parent:Do_Order()
+ end
else
self.Parent:Do_Order()
end
@@ -978,7 +984,110 @@ function multi:mainloop()
else
return "Already Running!"
end
- --print("Did you call multi:Stop()? This method should not be used when using multi:mainloop() unless of course you wanted to stop it! you can restart the multi, by using multi:reboot() and calling multi:mainloop() again or by using multi:uManager()")
+end
+function multi:protectedMainloop()
+ multi:protect()
+ if not multi.isRunning then
+ multi.isRunning=true
+ for i=1,#self.Tasks do
+ self.Tasks[i](self)
+ end
+ rawset(self,'Start',self.clock())
+ while self.Active do
+ self:Do_Order()
+ end
+ else
+ return "Already Running!"
+ end
+end
+function multi:unprotectedMainloop()
+ multi:unProtect()
+ if not multi.isRunning then
+ multi.isRunning=true
+ for i=1,#self.Tasks do
+ self.Tasks[i](self)
+ end
+ rawset(self,'Start',self.clock())
+ while self.Active do
+ local Loop=self.Mainloop
+ _G.ID=0
+ for _D=#Loop,1,-1 do
+ if Loop[_D] then
+ if Loop[_D].Active then
+ Loop[_D].Id=_D
+ self.CID=_D
+ Loop[_D]:Act()
+ end
+ end
+ end
+ end
+ else
+ return "Already Running!"
+ end
+end
+function multi:prioritizedMainloop1()
+ multi:enablePriority()
+ if not multi.isRunning then
+ multi.isRunning=true
+ for i=1,#self.Tasks do
+ self.Tasks[i](self)
+ end
+ rawset(self,'Start',self.clock())
+ while self.Active do
+ local Loop=self.Mainloop
+ _G.ID=0
+ local PS=self
+ for _D=#Loop,1,-1 do
+ if Loop[_D] then
+ if (PS.PList[PS.PStep])%Loop[_D].Priority==0 then
+ if Loop[_D].Active then
+ Loop[_D].Id=_D
+ self.CID=_D
+ Loop[_D]:Act()
+ end
+ end
+ end
+ end
+ PS.PStep=PS.PStep+1
+ if PS.PStep>7 then
+ PS.PStep=1
+ end
+ end
+ else
+ return "Already Running!"
+ end
+end
+function multi:prioritizedMainloop2()
+ multi:enablePriority2()
+ if not multi.isRunning then
+ multi.isRunning=true
+ for i=1,#self.Tasks do
+ self.Tasks[i](self)
+ end
+ rawset(self,'Start',self.clock())
+ while self.Active do
+ local Loop=self.Mainloop
+ _G.ID=0
+ local PS=self
+ for _D=#Loop,1,-1 do
+ if Loop[_D] then
+ if (PS.PStep)%Loop[_D].Priority==0 then
+ if Loop[_D].Active then
+ Loop[_D].Id=_D
+ self.CID=_D
+ Loop[_D]:Act()
+ end
+ end
+ end
+ end
+ PS.PStep=PS.PStep+1
+ if PS.PStep>self.Priority_Idle then
+ PS.PStep=1
+ end
+ end
+ else
+ return "Already Running!"
+ end
end
function multi._tFunc(self,dt)
for i=1,#self.Tasks do
@@ -1494,7 +1603,7 @@ function multi:newThread(name,func)
end
multi:setDomainName("Threads")
multi:setDomainName("Globals")
-multi.scheduler=multi:newUpdater()
+multi.scheduler=multi:newLoop()
multi.scheduler.Type="scheduler"
function multi.scheduler:setStep(n)
self.skip=tonumber(n) or 24
@@ -1503,7 +1612,7 @@ multi.scheduler.skip=0
multi.scheduler.counter=0
multi.scheduler.Threads=multi:linkDomain("Threads")
multi.scheduler.Globals=multi:linkDomain("Globals")
-multi.scheduler:OnUpdate(function(self)
+multi.scheduler:OnLoop(function(self)
self.counter=self.counter+1
for i=#self.Threads,1,-1 do
ret={}
@@ -1557,7 +1666,6 @@ multi.scheduler:OnUpdate(function(self)
end
end
end)
-multi.scheduler:setStep()
multi.scheduler:Pause()
multi.OnError=multi:newConnection()
function multi:newThreadedAlarm(name,set)
diff --git a/examples/love2d Threading Example/multi/integration/lanesManager.lua b/examples/love2d Threading Example/multi/integration/lanesManager.lua
index 8d07bf1..9c0e60b 100644
--- a/examples/love2d Threading Example/multi/integration/lanesManager.lua
+++ b/examples/love2d Threading Example/multi/integration/lanesManager.lua
@@ -125,12 +125,9 @@ function multi:newSystemThread(name,func)
end)
return c
end
-print("Intergrated Lanes!")
-multi.intergration={} -- for module creators
-multi.intergration.GLOBAL=GLOBAL
-multi.intergration.THREAD=THREAD
-multi.intergration.lanes={}
-multi.intergration.lanes.GLOBAL=GLOBAL -- for module creators
-multi.intergration.lanes.THREAD=THREAD -- for module creators
-require("multi.intergration.shared.shared")
+print("Integrated Lanes!")
+multi.integration={} -- for module creators
+multi.integration.GLOBAL=GLOBAL
+multi.integration.THREAD=THREAD
+require("multi.integration.shared.shared")
return {init=function() return GLOBAL,THREAD end}
diff --git a/examples/love2d Threading Example/multi/integration/loveManager.lua b/examples/love2d Threading Example/multi/integration/loveManager.lua
index 7a332ab..2440ade 100644
--- a/examples/love2d Threading Example/multi/integration/loveManager.lua
+++ b/examples/love2d Threading Example/multi/integration/loveManager.lua
@@ -2,14 +2,14 @@ require("multi.compat.love2d")
function multi:canSystemThread()
return true
end
-multi.intergration={}
-multi.intergration.love2d={}
-multi.intergration.love2d.ThreadBase=[[
+multi.integration={}
+multi.integration.love2d={}
+multi.integration.love2d.ThreadBase=[[
__THREADNAME__=({...})[1]
require("love.filesystem")
require("love.system")
require("love.timer")
-require("multi.all")
+require("multi")
GLOBAL={}
setmetatable(GLOBAL,{
__index=function(t,k)
@@ -29,6 +29,7 @@ setmetatable(GLOBAL,{
function __sync__()
local data=__mythread__:pop()
while data do
+ love.timer.sleep(.001)
if type(data)=="string" then
local cmd,tp,name,d=data:match("(%S-) (%S-) (%S-) (.+)")
if name=="__DIEPLZ"..__THREADNAME__.."__" then
@@ -79,7 +80,7 @@ function resolveType(tp,d)
elseif tp=="bool" then
return (d=="true")
elseif tp=="function" then
- return loadDump(d)
+ return loadDump("[==["..d.."]==]")
elseif tp=="table" then
return loadstring("return "..d)()
elseif tp=="nil" then
@@ -91,11 +92,11 @@ end
function resolveData(v)
local data=""
if type(v)=="table" then
- data=ToStr(v)
+ return ToStr(v)
elseif type(v)=="function" then
- data=dump(v)
+ return dump(v)
elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then
- data=tostring(v)
+ return tostring(v)
end
return data
end
@@ -152,24 +153,16 @@ end
function sThread.hold(n)
repeat __sync__() until n()
end
-updater=multi:newUpdater()
-updater:OnUpdate(function(self)
- local data=__mythread__:pop()
- while data do
- if type(data)=="string" then
- local cmd,tp,name,d=data:match("(%S-) (%S-) (%S-) (.+)")
- if name=="__DIEPLZ"..__THREADNAME__.."__" then
- error("Thread: "..__THREADNAME__.." has been stopped!")
- end
- if cmd=="SYNC" then
- __proxy__[name]=resolveType(tp,d)
- end
- else
- __proxy__[name]=data
- end
- data=__mythread__:pop()
+multi:newLoop(function(self)
+ self:Pause()
+ local ld=multi:getLoad()
+ self:Resume()
+ if ld<80 then
+ love.timer.sleep(.01)
end
end)
+updater=multi:newUpdater()
+updater:OnUpdate(__sync__)
func=loadDump([=[INSERT_USER_CODE]=])()
multi:mainloop()
]]
@@ -191,7 +184,7 @@ setmetatable(GLOBAL,{
end,
})
THREAD={} -- Allow main thread to interact with these objects as well
-multi.intergration.love2d.mainChannel=love.thread.getChannel("__MainChan__")
+multi.integration.love2d.mainChannel=love.thread.getChannel("__MainChan__")
function ToStr(val, name, skipnewlines, depth)
skipnewlines = skipnewlines or false
depth = depth or 0
@@ -228,7 +221,7 @@ function resolveType(tp,d)
elseif tp=="bool" then
return (d=="true")
elseif tp=="function" then
- return loadDump(d)
+ return loadDump("[==["..d.."]==]")
elseif tp=="table" then
return loadstring("return "..d)()
elseif tp=="nil" then
@@ -240,11 +233,11 @@ end
function resolveData(v)
local data=""
if type(v)=="table" then
- data=ToStr(v)
+ return ToStr(v)
elseif type(v)=="function" then
- data=dump(v)
+ return dump(v)
elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then
- data=tostring(v)
+ return tostring(v)
end
return data
end
@@ -274,10 +267,10 @@ function multi:newSystemThread(name,func) -- the main method
local c={}
c.name=name
c.ID=c.name..""
- c.thread=love.thread.newThread(multi.intergration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func)))
+ c.thread=love.thread.newThread(multi.integration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func)))
c.thread:start(c.ID)
function c:kill()
- multi.intergration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__"
+ multi.integration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__"
end
return c
end
@@ -310,11 +303,11 @@ function THREAD.hold(n)
multi.OBJ_REF:Resume()
end
__channels__={}
-multi.intergration.GLOBAL=GLOBAL
-multi.intergration.THREAD=THREAD
+multi.integration.GLOBAL=GLOBAL
+multi.integration.THREAD=THREAD
updater=multi:newUpdater()
updater:OnUpdate(function(self)
- local data=multi.intergration.love2d.mainChannel:pop()
+ local data=multi.integration.love2d.mainChannel:pop()
while data do
--print("MAIN:",data)
if type(data)=="string" then
@@ -342,19 +335,21 @@ updater:OnUpdate(function(self)
else
__proxy__[name]=data
end
- data=multi.intergration.love2d.mainChannel:pop()
+ data=multi.integration.love2d.mainChannel:pop()
end
end)
-require("multi.intergration.shared.shared")
-print("Intergrated Love2d!")
+require("multi.integration.shared.shared")
+print("Integrated Love2d!")
return {
init=function(t)
if t then
if t.threadNamespace then
- multi.intergration.love2d.ThreadBase:gsub("sThread",t.threadNamespace)
+ multi.integration.THREADNAME=t.threadNamespace
+ multi.integration.love2d.ThreadBase:gsub("sThread",t.threadNamespace)
end
if t.globalNamespace then
- multi.intergration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace)
+ multi.integration.GLOBALNAME=t.globalNamespace
+ multi.integration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace)
end
end
return GLOBAL,THREAD
diff --git a/examples/love2d Threading Example/multi/integration/shared/shared.lua b/examples/love2d Threading Example/multi/integration/shared/shared.lua
index 06bb8cf..92a1b44 100644
--- a/examples/love2d Threading Example/multi/integration/shared/shared.lua
+++ b/examples/love2d Threading Example/multi/integration/shared/shared.lua
@@ -280,7 +280,6 @@ if love then
GLOBAL["THREADED_JQ"]=nil -- remove it
GLOBAL["THREADED_JQO"]=nil -- remove it
multi:newLoop(function()
- sThread.sleep(.001) -- lets allow cpu time for other processes on our system!
local job=JQI:pop()
if job then
local ID=table.remove(job,1) -- return and remove
diff --git a/multi/init.lua b/multi/init.lua
index 42f8b74..feadba6 100644
--- a/multi/init.lua
+++ b/multi/init.lua
@@ -108,11 +108,13 @@ function multi:setThrestimed(n)
end
function multi:getLoad()
return multi:newFunction(function(self)
+ multi.scheduler:Pause()
local sample=#multi.Mainloop
local FFloadtest=0
multi:benchMark(multi.threstimed):OnBench(function(_,l3) FFloadtest=l3*(1/multi.threstimed) end)
self:hold(function() return FFloadtest~=0 end)
local val=FFloadtest/sample
+ multi.scheduler:Resume()
if val>multi.threshold then
return 0
else
@@ -617,7 +619,11 @@ function multi:hold(task)
env:OnEvent(function(envt) envt:Pause() envt.Active=false end)
while env.Active do
if love then
- self.Parent:lManager()
+ if love.graphics then
+ self.Parent:lManager()
+ else
+ self.Parent:Do_Order()
+ end
else
self.Parent:Do_Order()
end
diff --git a/multi/integration/loveManager.lua b/multi/integration/loveManager.lua
index 0a144d9..2440ade 100644
--- a/multi/integration/loveManager.lua
+++ b/multi/integration/loveManager.lua
@@ -153,6 +153,14 @@ end
function sThread.hold(n)
repeat __sync__() until n()
end
+multi:newLoop(function(self)
+ self:Pause()
+ local ld=multi:getLoad()
+ self:Resume()
+ if ld<80 then
+ love.timer.sleep(.01)
+ end
+end)
updater=multi:newUpdater()
updater:OnUpdate(__sync__)
func=loadDump([=[INSERT_USER_CODE]=])()
diff --git a/multi/integration/shared/shared.lua b/multi/integration/shared/shared.lua
index 06bb8cf..92a1b44 100644
--- a/multi/integration/shared/shared.lua
+++ b/multi/integration/shared/shared.lua
@@ -280,7 +280,6 @@ if love then
GLOBAL["THREADED_JQ"]=nil -- remove it
GLOBAL["THREADED_JQO"]=nil -- remove it
multi:newLoop(function()
- sThread.sleep(.001) -- lets allow cpu time for other processes on our system!
local job=JQI:pop()
if job then
local ID=table.remove(job,1) -- return and remove