diff --git a/changes.md b/changes.md index b5340d6..7781925 100644 --- a/changes.md +++ b/changes.md @@ -1,22 +1,29 @@ #Changes [TOC] -Update: 2.0.0 Big update +Update: 2.0.0 Big update (Lots of additions some changes) ------------------------ -**Note:** After doing some testing, I have noticed that using multi-objects are slightly, quite a bit, faster than using (coroutines)multi:newthread(). Only create a thread if there is no other possibility! System threads are different and will improve performance if you know what you are doing. Using a (coroutine)thread as a loop with a timer is slower than using a TLoop! If you do not need the holding features I strongly recommend that you use the multi-objects. This could be due to the scheduler that I am using, and I am looking into improving the performance of the scheduler for (coroutine)threads. This is still a work in progress so expect things to only get better as time passes! +**Note:** ~~After doing some testing, I have noticed that using multi-objects are slightly, quite a bit, faster than using (coroutines)multi:newthread(). Only create a thread if there is no other possibility! System threads are different and will improve performance if you know what you are doing. Using a (coroutine)thread as a loop with a timer is slower than using a TLoop! If you do not need the holding features I strongly recommend that you use the multi-objects. This could be due to the scheduler that I am using, and I am looking into improving the performance of the scheduler for (coroutine)threads. This is still a work in progress so expect things to only get better as time passes!~~ This was the reason threadloop was added. It binds the thread scheduler into the mainloop allowing threads to run much faster than before. Also the use of locals is now possible since I am not dealing with seperate objects. And finally reduced function overhead helps keep the threads running better. #Added: - `nGLOBAL = require("multi.integration.networkManager").init()` - `node = multi:newNode(tbl: settings)` - `master = multi:newMaster(tbl: settings)` - `multi:nodeManager(port)` +- `thread.isThread()` -- for coroutine based threads +- New setting to the main loop,stopOnError which defaults to true. This will cause the object that crashes when under protect to be destroyed, so the error does not keep happening. +- multi:threadloop(settings) works just like mainloop, but prioritizes (corutine based) threads. Regular multi-objects will still work. This improves the preformance of (coroutine based) threads greatly. -Node: +Changed: +- When a (corutine based)thread errors it does not print anymore! Conect to multi.OnError() to get errors when they happen! + +#Node: - node:sendTo(name,data) - node:pushTo(name,data) - node:peek() - node:pop() +- node:getConsole() -Master: +#Master: - master:doToAll(func) - master:register(name,node,func) - master:execute(name,node,...) @@ -27,10 +34,20 @@ Master: - master:pushTo(name,data) - master:peek() - master:pop() +- master:OnError(nodename, error) -- if a node has an error this is triggered. -**Note On Queues:** When it comes to network queues, they only send 1 way. What I mean by that is that if the master sends a message to a node, its own queue will not get populated at all. The reason for this is because syncing between which popped from what network queue would make things really slow and would not perform so well. This means you have to code a bit differently. Use: master getFreeNode() to get the name of the node under the least amount of load. Then handle the sending of data to each node that way. +#Going forward: +- Improve Performance +- Fix supporting libraries (Bin, and net need tons of work) +- Look for the bugs +- Figure out what I can do to make this library more awesome -**Note:** These examples assume that you have already connected the nodes to the node manager. Also you do not need to use the node manager, but sometimes broadcast does not work as expected and the master doesnot connect to the nodes. Using the node manager offers nice features like: removing nodes from the master when they have disconnected, and automatically telling the master when nodes have been added. + +**Note On Queues:** When it comes to network queues, they only send 1 way. What I mean by that is that if the master sends a message to a node, its own queue will not get populated at all. The reason for this is because syncing between which popped from what network queue would make things really slow and would not perform well at all. This means you have to code a bit differently. Use: master getFreeNode() to get the name of the node under the least amount of load. Then handle the sending of data to each node that way. + +Now there is a little trick you can do. If you combine both networkmanager and systemthreading manager, then you could have a proxy queue for all system threads that can pull from that "node". Now data passing within a lan network, (And wan network if using the node manager, though p2p isn't working as i would like and you would need to open ports and make things work. Remember you can define an port for your node so you can port forward that if you want), is fast enough, but the waiting problem is something to consider. Ask yourseld what you are coding and if network paralisim is worth using. + +**Note:** These examples assume that you have already connected the nodes to the node manager. Also you do not need to use the node manager, but sometimes broadcast does not work as expected and the master doesnot connect to the nodes. Using the node manager offers nice features like: removing nodes from the master when they have disconnected, and automatically telling the master when nodes have been added. A more complete example showing connections regardless of order will be shown in the example folder check it out. New naming scheme too. **NodeManager.lua** ```lua @@ -49,6 +66,8 @@ multi:mainloop(settings) ``` +Side note: I had a setting called cross talk that would allow nodes to talk to each other. After some tought I decided to not allow nodes to talk to each other directly! You however can create another master withing the node. (The node will connect to its own master as well). This will give you the ability "Cross talk" with each node. Reimplementing the master features into each node directly was un nessacery. + **Node.lua** ```lua package.path="?/init.lua;?.lua;"..package.path @@ -56,7 +75,6 @@ multi = require("multi") local GLOBAL, THREAD = require("multi.integration.lanesManager").init() nGLOBAL = require("multi.integration.networkManager").init() master = multi:newNode{ - crossTalk = false, -- default value, allows nodes to talk to eachother. WIP NOT READY YET! allowRemoteRegistering = true, -- allows you to register functions from the master on the node, default is false name = nil, -- default value noBroadCast = true, -- if using the node manager, set this to true to prevent the node from broadcasting diff --git a/multi/init.lua b/multi/init.lua index 1d67f9f..c5d3bf8 100644 --- a/multi/init.lua +++ b/multi/init.lua @@ -6,7 +6,7 @@ Copyright (c) 2017 Ryan Ward Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +to use, copy, modify, merge, publish, distribute, sub-license, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: @@ -750,15 +750,83 @@ function multi:newCondition(func) return c end multi.NewCondition=multi.newCondition +function multi:threadloop(settings) + multi.scheduler:Destroy() -- destroy is an interesting thing... if you dont set references to nil, then you only remove it from the mainloop + local Threads=multi:linkDomain("Threads") + local Globals=multi:linkDomain("Globals") + local counter=0 + local tick = 0 + while true do + tick = tick + 1 + if tick == 1024 then + tick = 0 + multi:uManager(settings) + end + counter=counter+1 + for i=#Threads,1,-1 do + ret={} + if coroutine.status(Threads[i].thread)=="dead" then + table.remove(Threads,i) + else + if Threads[i].timer:Get()>=Threads[i].sleep then + if Threads[i].firstRunDone==false then + Threads[i].firstRunDone=true + Threads[i].timer:Start() + _,ret=coroutine.resume(Threads[i].thread,Threads[i].ref) + else + _,ret=coroutine.resume(Threads[i].thread,Globals) + end + if _==false then + self.Parent.OnError:Fire(Threads[i],"Error in thread: <"..Threads[i].Name.."> "..ret) + end + if ret==true or ret==false then + ret={} + end + end + if ret then + if ret[1]=="_kill_" then + table.remove(Threads,i) + elseif ret[1]=="_sleep_" then + Threads[i].timer:Reset() + Threads[i].sleep=ret[2] + elseif ret[1]=="_skip_" then + Threads[i].timer:Reset() + Threads[i].sleep=math.huge + local event=multi:newEvent(function(evnt) return counter>=evnt.counter end) + event.link=Threads[i] + event.counter=counter+ret[2] + event:OnEvent(function(evnt) + evnt.link.sleep=0 + end) + elseif ret[1]=="_hold_" then + Threads[i].timer:Reset() + Threads[i].sleep=math.huge + local event=multi:newEvent(ret[2]) + event.link=Threads[i] + event:OnEvent(function(evnt) + evnt.link.sleep=0 + end) + elseif ret.Name then + Globals[ret.Name]=ret.Value + end + end + end + end + end +end function multi:mainloop(settings) multi.defaultSettings = settings or multi.defaultSettings if not multi.isRunning then local protect = false local priority = false + local stopOnError = true if settings then if settings.preLoop then settings.preLoop(self) end + if settings.stopOnError then + stopOnError = settings.stopOnError + end protect = settings.protect priority = settings.priority end @@ -781,6 +849,9 @@ function multi:mainloop(settings) if err then Loop[_D].error=err self.OnError:Fire(Loop[_D],err) + if stopOnError then + Loop[_D]:Destroy() + end end end end @@ -803,6 +874,9 @@ function multi:mainloop(settings) if err then Loop[_D].error=err self.OnError:Fire(Loop[_D],err) + if stopOnError then + Loop[_D]:Destroy() + end end end end @@ -826,6 +900,9 @@ function multi:mainloop(settings) if err then Loop[_D].error=err self.OnError:Fire(Loop[_D],err) + if stopOnError then + Loop[_D]:Destroy() + end end end end @@ -1386,6 +1463,9 @@ end function thread.yeild() coroutine.yield({"_sleep_",0}) end +function thread.isThread() + return coroutine.running() +end function thread.getCores() return thread.__CORES end @@ -1488,8 +1568,7 @@ multi.scheduler:OnLoop(function(self) _,ret=coroutine.resume(self.Threads[i].thread,self.Globals) end if _==false then - self.Parent.OnError:Fire(self.Threads[i],ret) - print("Error in thread: <"..self.Threads[i].Name.."> "..ret) + self.Parent.OnError:Fire(Threads[i],"Error in thread: <"..Threads[i].Name.."> "..ret) end if ret==true or ret==false then print("Thread Ended!!!") diff --git a/multi/integration/networkManager.lua b/multi/integration/networkManager.lua index 4928d0d..3316358 100644 --- a/multi/integration/networkManager.lua +++ b/multi/integration/networkManager.lua @@ -1,4 +1,4 @@ --- CURRENT TASK: newNetThread() +-- CURRENT TASK: local multi = require("multi") local net = require("net") @@ -158,7 +158,7 @@ function multi:newNode(settings) local node = {} node.name = name multi.OnError(function(i,error) - node.OnError(node,error,node.server) + node.OnError:Fire(node,error,node.server) end) node.server = net:newUDPServer(0) -- hosts the node using the default port _, node.port = node.server.udp:getsockname() @@ -168,12 +168,16 @@ function multi:newNode(settings) node.hasFuncs = {} node.OnError = multi:newConnection() node.OnError(function(node,err,master) + print("ERROR",err,node.name) local temp = bin.new() - temp:addBlock(#node.name) + temp:addBlock(#node.name,2) temp:addBlock(node.name) - temp:addBlock(#err) + temp:addBlock(#err,2) temp:addBlock(err) - node.server:send(char(CMD_ERROR..temp)) + for i,v in pairs(node.connections) do + print(i) + v[1]:send(v[2],char(CMD_ERROR)..temp.data,v[3]) + end end) if settings.managerDetails then local c = net:newTCPClient(settings.managerDetails[1],settings.managerDetails[2]) @@ -216,14 +220,17 @@ function multi:newNode(settings) end function node:getConsole() local c = {} - c.connections = node.connections - function c:print(...) + local conn = node.connections + function c.print(...) local data = char(CMD_CONSOLE)..packData({...}) - for i,v in pairs(self.connections) do + for i,v in pairs(conn) do + --print(i) v[1]:send(v[2],data,v[3]) end - print("sent message") end + -- function c:printTo() + + -- end return c end node.loadRate=1 @@ -266,7 +273,10 @@ function multi:newNode(settings) local func = holder:getBlock("s",len2) args = resolveData(args) func = resolveData(func) - func(unpack(args)) + status, err = pcall(func,node,unpack(args)) + if not status then + node.OnError:Fire(node,err,server) + end elseif cmd == CMD_INITNODE then print("Connected with another node!") node.connections[dat]={server,ip,port} @@ -427,7 +437,7 @@ function multi:newMaster(settings) -- You will be able to have more than one mas multi:newTLoop(function(loop) if self.connections[name]~=nil then self.connections[name]:send(data) - loop:Desrtoy() + loop:Destroy() end end,.1) else @@ -486,7 +496,7 @@ function multi:newMaster(settings) -- You will be able to have more than one mas local node = temp:getBlock("s",len) len = temp:getBlock("n",2) local err = temp:getBlock("s",len) - master.OnError(name,err) + master.OnError:Fire(name,err) elseif cmd == CMD_CONSOLE then print(unpack(resolveData(dat))) elseif cmd == CMD_PONG then diff --git a/node.lua b/node.lua index 11f4cef..188af56 100644 --- a/node.lua +++ b/node.lua @@ -10,7 +10,7 @@ node = multi:newNode{ --managerDetails = {"localhost",12345}, -- connects to the node manager if one exists } function RemoteTest(a,b,c) -- a function that we will be executing remotely - print("Yes I work!",a,b,c) + --print("Yes I work!",a,b,c) multi:newThread("waiter",function() print("Hello!") while true do @@ -31,6 +31,7 @@ multi:newThread("some-test",function() end,"NODE_TESTNODE") settings = { priority = 0, -- 1 or 2 - protect = false, -- if something goes wrong we will crash hard, but the speed gain is good + stopOnError = true, + protect = true, -- if something goes wrong we will crash hard, but the speed gain is good } multi:mainloop(settings) diff --git a/test.lua b/test.lua index f00ceb5..05b4126 100644 --- a/test.lua +++ b/test.lua @@ -11,22 +11,25 @@ master = multi:newMaster{ --managerDetails = {"localhost",12345}, -- the details to connect to the node manager (ip,port) } -- Send to all the nodes that are connected to the master -master.OnNodeConnected(function(node) +master.OnError(function(name,err) + print(name.." has encountered an error: "..err) +end) +master.OnNodeConnected(function(name) print("Lets Go!") - master:newNetworkThread("Thread",function() - local node = _G.node - local console = node:getConsole() + master:newNetworkThread("Thread",function(node) + local print = node:getConsole().print -- it is important to define things as local... another thing i could do is fenv to make sure all masters work in a protectd isolated enviroment multi:newTLoop(function() - console:print("Yo whats up man!") + print("Yo whats up man!") + error("doing a test") end,1) end) - master:execute("RemoteTest",node,1,2,3) + master:execute("RemoteTest",name,1,2,3) multi:newThread("waiter",function() - print("Hello!",node) + print("Hello!",name) while true do thread.sleep(2) - master:pushTo(node,"This is a test 2") - if master.connections["NODE_"..node]==nil then + master:pushTo(name,"This is a test 2") + if master.connections["NODE_"..name]==nil then thread.kill() end end @@ -45,6 +48,7 @@ end,"NODE_TESTNODE") -- Starting the multitasker settings = { priority = 0, -- 0, 1 or 2 - protect = false, + protect = true, } -multi:mainloop(settings) +multi:threadloop(settings) +--multi:mainloop(settings)