diff --git a/changes.html b/changes.html index 55c2752..235c3f4 100644 --- a/changes.html +++ b/changes.html @@ -9,7 +9,382 @@ -

Changes

Update: 1.11.1
Love2d change:
I didn’t make a mistake but didn’t fully understand how the new love.run function worked.
So, it works by returning a function that allows for running the mainloop. So, this means that we can do something like this:

Changes

+
Update: 12.0.0 Big update (Lots of additions some changes)
Note: After doing some testing, I have noticed that using multi-objects are slightly, quite a bit, faster than using (coroutines)multi:newthread(). Only create a thread if there is no other possibility! System threads are different and will improve performance if you know what you are doing. Using a (coroutine)thread as a loop with a timer is slower than using a TLoop! If you do not need the holding features I strongly recommend that you use the multi-objects. This could be due to the scheduler that I am using, and I am looking into improving the performance of the scheduler for (coroutine)threads. This is still a work in progress so expect things to only get better as time passes! This was the reason threadloop was added. It binds the thread scheduler into the mainloop allowing threads to run much faster than before. Also the use of locals is now possible since I am not dealing with seperate objects. And finally reduced function overhead helps keep the threads running better.
Added:
Changed:
Node:
Master:
Bugs
Going forward:
Note On Queues: When it comes to network queues, they only send 1 way. What I mean by that is that if the master sends a message to a node, its own queue will not get populated at all. The reason for this is because syncing between which popped from what network queue would make things really slow and would not perform well at all. This means you have to code a bit differently. Use: master getFreeNode() to get the name of the node under the least amount of load. Then handle the sending of data to each node that way.
Now there is a little trick you can do. If you combine both networkmanager and systemthreading manager, then you could have a proxy queue for all system threads that can pull from that “node”. Now data passing within a lan network, (And wan network if using the node manager, though p2p isn’t working as i would like and you would need to open ports and make things work. Remember you can define an port for your node so you can port forward that if you want), is fast enough, but the waiting problem is something to consider. Ask yourseld what you are coding and if network paralisim is worth using.
Note: These examples assume that you have already connected the nodes to the node manager. Also you do not need to use the node manager, but sometimes broadcast does not work as expected and the master doesnot connect to the nodes. Using the node manager offers nice features like: removing nodes from the master when they have disconnected, and automatically telling the master when nodes have been added. A more complete example showing connections regardless of order will be shown in the example folder check it out. New naming scheme too.
NodeManager.lua
package.path="?/init.lua;?.lua;"..package.path
+multi = require("multi")
+local GLOBAL, THREAD = require("multi.integration.lanesManager").init()
+nGLOBAL = require("multi.integration.networkManager").init()
+multi:nodeManager(12345) -- Host a node manager on port: 12345
+print("Node Manager Running...")
+settings = {
+    priority = 0, -- 1 or 2
+    protect = false,
+}
+multi:mainloop(settings)
+-- Thats all you need to run the node manager, everything else is done automatically
+
Side note: I had a setting called cross talk that would allow nodes to talk to each other. After some tought I decided to not allow nodes to talk to each other directly! You however can create another master withing the node. (The node will connect to its own master as well). This will give you the ability “Cross talk” with each node. Reimplementing the master features into each node directly was un nessacery. 
Node.lua
package.path="?/init.lua;?.lua;"..package.path
+multi = require("multi")
+local GLOBAL, THREAD = require("multi.integration.lanesManager").init()
+nGLOBAL = require("multi.integration.networkManager").init()
+master = multi:newNode{
+    allowRemoteRegistering = true, -- allows you to register functions from the master on the node, default is false
+    name = nil, -- default value
+    noBroadCast = true, -- if using the node manager, set this to true to prevent the node from broadcasting
+    managerDetails = {"localhost",12345}, -- connects to the node manager if one exists
+}
+function RemoteTest(a,b,c) -- a function that we will be executing remotely
+    print("Yes I work!",a,b,c)
+end
+settings = {
+    priority = 0, -- 1 or 2
+    protect = false, -- if something goes wrong we will crash hard, but the speed gain is good
+}
+multi:mainloop(settings)
+
Master.lua
-- set up the package
+package.path="?/init.lua;?.lua;"..package.path
+-- Import the libraries
+multi = require("multi")
+local GLOBAL, THREAD = require("multi.integration.lanesManager").init()
+nGLOBAL = require("multi.integration.networkManager").init()
+-- Act as a master node
+master = multi:newMaster{
+    name = "Main", -- the name of the master
+    noBroadCast = true, -- if using the node manager, set this to true to avoid double connections
+    managerDetails = {"localhost",12345}, -- the details to connect to the node manager (ip,port)
+}
+-- Send to all the nodes that are connected to the master
+master:doToAll(function(node_name)
+    master:register("TestFunc",node_name,function(msg)
+        print("It works: "..msg)
+    end)
+    multi:newAlarm(2):OnRing(function(alarm)
+        master:execute("TestFunc",node_name,"Hello!")
+        alarm:Destroy()
+    end)
+    multi:newThread("Checker",function()
+        while true do
+            thread.sleep(1)
+            if nGLOBAL["test"] then
+                print(nGLOBAL["test"])
+                thread.kill()
+            end
+        end
+    end)
+    nGLOBAL["test2"]={age=22}
+end)
+
+-- Starting the multitasker
+settings = {
+    priority = 0, -- 0, 1 or 2
+    protect = false,
+}
+multi:mainloop(settings)
+
Note: There are many ways to work this. You could send functions/methods to a node like haw systemThreadedJobQueue work. Or you could write the methods you want in advance in each node file and send over the command to run the method with arguments … and it will return the results. Network threading is different than system threading. Data transfer is really slow compared to system threading. In fact the main usage for this feature in the library is mearly for experments. Right now I honestly do not know what I want to do with this feature and what I am going to add to this feature. The ablitiy to use this frature like a system thread will be possible, but there are some things that differ.
Changed:
Modifying the global stack is not the best way to manage or load in the library.
-- Base Library
+multi = require("multi")
+-- In Lanes
+multi = require("multi")
+local GLOBAL, THREAD = require("multi.integration.lanesManager").init()
+-- In Love2d
+multi = require("multi")
+GLOBAL, THREAD = require("multi.integration.loveManager").init()
+-- In Luvit
+local timer = require("timer")
+local thread = require("thread")
+multi = require("multi")
+require("multi.integration.luvitManager").init(thread,timer) -- Luvit does not cuttently have support for the global table or threads.
+
Improvements:
Removed:
The new settings table makes all of these possible and removes a lot of function overhead that was going on before.
multi:mainloop{
+    priority = 1, -- 1 or 2
+    protect = true, -- Should I use pcall to ignore errors?
+    preLoop = function(self) -- a function that is called before the mainloop does its thing
+        multi:newTLoop(function()
+            print("Hello whats up!")
+            error(":P")
+        end,1)
+        multi.OnError(function(obj,err)
+            print(err)
+            obj:Destroy()
+        end)
+    end,
+}
+
Update: 1.11.1
Love2d change:
I didn’t make a mistake but didn’t fully understand how the new love.run function worked.
So, it works by returning a function that allows for running the mainloop. So, this means that we can do something like this:
function love.draw()
     multi.dManager() -- If using my guimanager, if not omit this
 end
-
Update: 1.10.0
Note: The library is now considered to be stable!
Upcoming: Network parallelism is on the way. It is in the works and should be released soon
Added:

Update: 1.10.0

Note: The library is now considered to be stable!
Upcoming: Network parallelism is on the way. It is in the works and should be released soon

Added:

Example of threaded connections

function1
     connOut:Fire("Test From Main Thread: "..a.."\n")
 end,1)
-

Fixed:

loveManager and shared threading objects

Update: 1.4.1 - First Public release of the library

IMPORTANT:
Every update I make aims to make things simpler more efficient and just better, but a lot of old code, which can be really big, uses a lot of older features. I know the pain of having to rewrite everything. My promise to my library users is that I will always have backwards support for older features! New ways may exist that are quicker and easier, but the old features/methods will be supported.
Note: Version 2.x.x sort of breaks this promise. Sorry about that, but a new major version means changes that had to be made. Not too much has changed though and base code is 100% compatiable. What changed was how you init the library and some files that were removed due to not really being used by what i have seen. The older backwards compat file was for an older version of the library that was changed before the public release had any traction. The goal is still to provide a easy way to multitask in lua. I’ll try my best however to ensure that not much changes and that changes are easy to make if they are introduced.

diff --git a/changes.md b/changes.md index 05a5c69..071b970 100644 --- a/changes.md +++ b/changes.md @@ -1,6 +1,6 @@ #Changes [TOC] -Update: 2.0.0 Big update (Lots of additions some changes) +Update: 12.0.0 Big update (Lots of additions some changes) ------------------------ **Note:** ~~After doing some testing, I have noticed that using multi-objects are slightly, quite a bit, faster than using (coroutines)multi:newthread(). Only create a thread if there is no other possibility! System threads are different and will improve performance if you know what you are doing. Using a (coroutine)thread as a loop with a timer is slower than using a TLoop! If you do not need the holding features I strongly recommend that you use the multi-objects. This could be due to the scheduler that I am using, and I am looking into improving the performance of the scheduler for (coroutine)threads. This is still a work in progress so expect things to only get better as time passes!~~ This was the reason threadloop was added. It binds the thread scheduler into the mainloop allowing threads to run much faster than before. Also the use of locals is now possible since I am not dealing with seperate objects. And finally reduced function overhead helps keep the threads running better. @@ -17,14 +17,16 @@ Update: 2.0.0 Big update (Lots of additions some changes) Changed: - When a (corutine based)thread errors it does not print anymore! Conect to multi.OnError() to get errors when they happen! - Connections get yet another update. Connect takes an additional argument now which is the position in the table that the func should be called. Note: Fire calls methods backwards so 1 is the back and the # of connections (the default value) is the beginning of the call table -- The love2d compat layer has now been revamped allowing module creators to connect to events without the user having to add likes of code for those events. Its all done automagically +- The love2d compat layer has now been revamped allowing module creators to connect to events without the user having to add likes of code for those events. Its all done automagically. +- This library is about 8 years old and using 2.0.0 makes it seem young. I changed it to 12.0.0 since it has some huge changes and there were indeed 12 major releases that added some cool things. Going forward I'll use major.minor.bugfix +- multi.OnError() is now required to capture errors that are thrown when in prorected mode. #Node: - node:sendTo(name,data) - node:pushTo(name,data) - node:peek() - node:pop() -- node:getConsole() +- node:getConsole() -- has only 1 function print which allows you to print to the master. #Master: - master:doToAll(func) @@ -40,9 +42,13 @@ Changed: - master:OnError(nodename, error) -- if a node has an error this is triggered. #Bugs -- Fixed a small typo I made which caused a hard crash when a (coroutine) thread crashes. This only happened if protect was false. Which is now the defualt value for speed reasons. +- Fixed a small typo I made which caused a hard crash when a (coroutine) thread crashes. This only happened if protect was true. #Going forward: +- I am really excited to finally get this update out there, but left one important thing out. enabling of enviroments for each master connected to a node. This would allow a node to isolate code from multiple masters so they cannot interact with each other. This will come out in version 12.1.0 But might take a while due to the job hunt that I am currently going through. +- Another feature that I am on the fence about is adding channels. They would work like queues, but are named so you can seperate the data from different channels where only one portion of can see certain data. +- I also might add a feature that allows different system threads to consume from a network queue if they are spaned on the same physical machine. This is possible at the moment, just doesn't have a dedicated object for handling this seamlessly. You can do this yourself though. +- Another feature that I am thinking of adding is crosstalk which is a setting that would allow nodes to talk to other nodes. I did not add it in this release since there are some issues that need to be worked out and its very messy atm. however since nodes are named. I may allow by default pushing data to another node, but not have the global table to sync since this is where the issue lies. - Improve Performance - Fix supporting libraries (Bin, and net need tons of work) - Look for the bugs diff --git a/test.lua b/examples/network-Master1.lua similarity index 85% rename from test.lua rename to examples/network-Master1.lua index 05b4126..f27e4b2 100644 --- a/test.lua +++ b/examples/network-Master1.lua @@ -7,7 +7,7 @@ nGLOBAL = require("multi.integration.networkManager").init() -- Act as a master node master = multi:newMaster{ name = "Main", -- the name of the master - --noBroadCast = true, -- if using the node manager, set this to true to avoid double connections + --noBroadCast = true, -- if using the node manager, set this to true to save on some cpu cycles --managerDetails = {"localhost",12345}, -- the details to connect to the node manager (ip,port) } -- Send to all the nodes that are connected to the master @@ -16,14 +16,14 @@ master.OnError(function(name,err) end) master.OnNodeConnected(function(name) print("Lets Go!") - master:newNetworkThread("Thread",function(node) + master:newNetworkThread("Thread",function(node) -- spawn a network thread on a node that can execute code and return date local print = node:getConsole().print -- it is important to define things as local... another thing i could do is fenv to make sure all masters work in a protectd isolated enviroment multi:newTLoop(function() print("Yo whats up man!") error("doing a test") end,1) end) - master:execute("RemoteTest",name,1,2,3) + master:execute("RemoteTest",name,1,2,3) -- calls a predefined or registered global method on a node multi:newThread("waiter",function() print("Hello!",name) while true do diff --git a/node.lua b/examples/network-node1.lua similarity index 85% rename from node.lua rename to examples/network-node1.lua index 188af56..4a828a5 100644 --- a/node.lua +++ b/examples/network-node1.lua @@ -5,8 +5,8 @@ nGLOBAL = require("multi.integration.networkManager").init() node = multi:newNode{ crossTalk = false, -- default value, allows nodes to talk to eachother. WIP NOT READY YET! allowRemoteRegistering = true, -- allows you to register functions from the master on the node, default is false - name = nil, --"TESTNODE", -- default value - --noBroadCast = true, -- if using the node manager, set this to true to prevent the node from broadcasting + name = nil, --"TESTNODE", -- default value is nil, if nil a random name is generated. Naming nodes are important if you assign each node on a network with a different task + --noBroadCast = true, -- if using the node manager, set this to true to save on some cpu cycles --managerDetails = {"localhost",12345}, -- connects to the node manager if one exists } function RemoteTest(a,b,c) -- a function that we will be executing remotely diff --git a/nodeManager.lua b/examples/network-nodeManager.lua similarity index 100% rename from nodeManager.lua rename to examples/network-nodeManager.lua diff --git a/multi/init.lua b/multi/init.lua index 0a1fc3e..f2211fd 100644 --- a/multi/init.lua +++ b/multi/init.lua @@ -23,8 +23,8 @@ SOFTWARE. ]] local bin = pcall(require,"bin") local multi = {} -multi.Version = "2.0.0" -multi._VERSION = "2.0.0" +multi.Version = "12.0.0" +multi._VERSION = "12.0.0" multi.stage = "stable" multi.__index = multi multi.Mainloop = {} diff --git a/rockspecs/multi-2.0-0.rockspec b/rockspecs/multi-12.0-0.rockspec similarity index 89% rename from rockspecs/multi-2.0-0.rockspec rename to rockspecs/multi-12.0-0.rockspec index c7c9f3d..1c531e8 100644 --- a/rockspecs/multi-2.0-0.rockspec +++ b/rockspecs/multi-12.0-0.rockspec @@ -1,8 +1,8 @@ package = "multi" -version = "1.11.0" +version = "12.0-0" source = { url = "git://github.com/rayaman/multi.git", - tag = "v1.11.0", + tag = "v12.0.0", } description = { summary = "Lua Multi tasking library", @@ -27,6 +27,7 @@ build = { ["multi.integration.lanesManager"] = "multi/integration/lanesManager.lua", ["multi.integration.loveManager"] = "multi/integration/loveManager.lua", ["multi.integration.luvitManager"] = "multi/integration/luvitManager.lua", + ["multi.integration.networkManager"] = "multi/integration/networkManager.lua", ["multi.integration.shared"] = "multi/integration/shared.lua" } } \ No newline at end of file diff --git a/samples/sample-master.lua b/samples/sample-master.lua new file mode 100644 index 0000000..03ebe05 --- /dev/null +++ b/samples/sample-master.lua @@ -0,0 +1,25 @@ +-- set up the package +package.path="?/init.lua;?.lua;"..package.path +-- Import the libraries +multi = require("multi") +local GLOBAL, THREAD = require("multi.integration.lanesManager").init() +nGLOBAL = require("multi.integration.networkManager").init() +-- Act as a master node +master = multi:newMaster{ + name = "Main", -- the name of the master + --noBroadCast = true, -- if using the node manager, set this to true to avoid double connections + --managerDetails = {"localhost",12345}, -- the details to connect to the node manager (ip,port) +} +master.OnError(function(name,err) + print(name.." has encountered an error: "..err) +end) +master.OnNodeConnected(function(name) + -- name is the name of the node that connected +end) +-- Starting the multitasker +settings = { + priority = 0, -- 0, 1 or 2 + protect = false, +} +multi:threadloop(settings) -- both mainloop and threadloop can be used. one pirotizes threads where the other pirotizes multiobjs +--multi:mainloop(settings) diff --git a/samples/sample-node.lua b/samples/sample-node.lua new file mode 100644 index 0000000..2e2dd3b --- /dev/null +++ b/samples/sample-node.lua @@ -0,0 +1,16 @@ +package.path="?/init.lua;?.lua;"..package.path +multi = require("multi") +local GLOBAL, THREAD = require("multi.integration.lanesManager").init() +nGLOBAL = require("multi.integration.networkManager").init() +node = multi:newNode{ + allowRemoteRegistering = true, -- allows you to register functions from the master on the node, default is false + name = nil, -- default value + --noBroadCast = true, -- if using the node manager, set this to true to prevent the node from broadcasting + --managerDetails = {"localhost",12345}, -- connects to the node manager if one exists +} +settings = { + priority = 0, -- 1 or 2 + stopOnError = true, -- if an actor crashes this will prevent it from constantly crashing over and over. You can leave this false and use multi.OnError to handle crashes as well + protect = true, -- always protect a node. Not really needed since all executed xode from a master is protected on execution to prevent issues. +} +multi:mainloop(settings) diff --git a/samples/sample-nodeManager.lua b/samples/sample-nodeManager.lua new file mode 100644 index 0000000..7d57596 --- /dev/null +++ b/samples/sample-nodeManager.lua @@ -0,0 +1,12 @@ +package.path="?/init.lua;?.lua;"..package.path +multi = require("multi") +local GLOBAL, THREAD = require("multi.integration.lanesManager").init() +nGLOBAL = require("multi.integration.networkManager").init() +multi:nodeManager(12345) -- Host a node manager on port: 12345 +print("Node Manager Running...") +settings = { + priority = 0, -- 1 or 2 + protect = false, +} +multi:mainloop(settings) +-- Thats all you need to run the node manager, everything else is done automatically