sigh processors have some bugs

working on more bugs... I estimate about 2 months of work for the next version haha
This commit is contained in:
Ryan Ward 2019-01-31 12:25:41 -05:00
parent 03c8f364b2
commit 0fd604e356
4 changed files with 123 additions and 70 deletions

View File

@ -633,9 +633,9 @@ Coroutine based Threading (CBT)
This was made due to the limitations of multiObj:hold(), which no longer exists. When this library was in its infancy and before I knew about coroutines, I actually tried to emulate what coroutines did in pure lua.
The threaded bariants of the non threaded objects do exist, but there isn't too much of a need to use them.
The main benefits of using the coroutine based threads it the thread.* namespace which gives you the ability to easily run code side by side.
The main benefits of using the coroutine based threads is the thread.* namespace which gives you the ability to easily run code side by side.
A quick not on how threads are managed in the library. The library contains a scheduler which keeps track of coroutines and manages them. Coroutines take some time then give off processing to another coroutine. Which means there are some methods that you need to use in order to hand off cpu time to other coroutines or the main thread.
A quick note on how threads are managed in the library. The library contains a scheduler which keeps track of coroutines and manages them. Coroutines take some time then give off processing to another coroutine. Which means there are some methods that you need to use in order to hand off cpu time to other coroutines or the main thread. You must hand off cpu time when inside of a non ending loop or your code will hang. Threads also have a slight delay before starting, about 3 seconds.
threads.*
---------

View File

@ -2,6 +2,9 @@
[TOC]
Update 13.0.0 So you documented it, finally! If I had a dollar for each time I found a bug working on 13.0.0 I'd be quite wealthy by now. How much lag could one expect when I've been coding with my own library wrong this entire time?
-------------
**Quick note** on the 13.0.0 update:
This update I went all in finding bugs and improving proformance within the library. I added some new features and the new task manager, which I used as a way to debug the library was a great help, so much so thats it is now a permanent feature. It's been about half a year since my last update, but so much work needed to be done. I hope you can find a use in your code to use my library. I am extremely proud of my work; 7 years of development, I learned so much about lua and programming through the creation of this library. It was fun, but there will always be more to add and bugs crawling there way in. I can't wait to see where this library goes in the future!
Fixed: Tons of bugs, I actually went through the entire library and did a full test of everything, I mean everything, while writing the documentation.
Changed:
- A few things, to make concepts in the library more clear.
@ -53,12 +56,13 @@ Fixed:
- Fixed an issue where any argument greater than 256^2/65536 bytes is sent the networkmanager would soft crash. This was fixed by increading the limit to 256^4/4294967296 bytes. The fix was changing a 2 to a 4. Arguments greater than 256^4 would be impossible in 32 bit lua, and highly unlikely even in lua 64 bit. Perhaps someone is reading an entire file into ram and then sending the entire file that they read over a socket for some reason all at once!?
Added:
- multi:newHyperThreadedProcess(STRING name) -- This is a version of the threaded process that gives each object created its own coroutine based thread which means you can use thread.* without affecting other objects created within the hyper threaded processes.
- Documentation, the purpose of 13.0.0, orginally going to be 12.2.3, but due to the amount of bugs and features I added couldn't become that. I actually still did my tests in the 12.2.3 branch in github.
- multi:newHyperThreadedProcess(STRING name) -- This is a version of the threaded process that gives each object created its own coroutine based thread which means you can use thread.* without affecting other objects created within the hyper threaded processes. Though, creating a self contained single thread is a better idea which when I eventually create the wiki page I'll discuss
- multi:newConnector() -- A simple object that allows you to use the new connection Fire syntax without using a multi obj or the standard object format that I follow.
- multi:purge() -- Removes all references to objects that are contained withing the processes list of tasks to do. Doing this will stop all objects from functioning. Calling Resume on an object should make it work again.
- multi:getTasksDetails(STRING format) -- Simple function, will get massive updates in the future, as of right now It will print out the current processes that are running; listing their type, uptime, and priority. More useful additions will be added in due time. Format can be either a string "s" or "t" see below for the table format
- multi:endTask(TID) -- Use multi:getTasksDetails("t") to get the tid of a task
- multi:enableLoadDetection() -- Since load detection puts some strain on the system (very little) I decided to make it something that has to be enabled. Once on it cant be turned off!
- multi:enableLoadDetection() -- Reworked how load detection works. It gives better values now, but it still needs some work before I am happy with it
```lua
package.path="?/init.lua;?.lua;"..package.path
@ -99,6 +103,11 @@ Table format for getTasksDetails(STRING format)
}
```
**Note:** After adding the getTasksDetails() function I noticed many areas where threads, and tasks were not being cleaned up and fixed the leaks. I also found out that a lot of tasks were starting by default and made them enable only. If you compare the benchmark from this version to last version you;ll notice a signifacant increase in performance.
**Going forward:**
- Add something
Update 12.2.2 Time for some more bug fixes!
-------------
Fixed: multi.Stop() not actually stopping due to the new pirority management scheme and preformance boost changes.

View File

@ -125,10 +125,59 @@ end
function multi:setThrestimed(n)
self.deltaTarget=n or .1
end
function multi:enableLoadDetection()
if multi.maxSpd then return end
-- here we are going to run a quick benchMark solo
local temp = multi:newProcessor()
temp:Start()
local t = os.clock()
local stop = false
temp:benchMark(.01):OnBench(function(time,steps)
stop = steps*1.1
end)
while not stop do
temp:uManager()
end
temp:__Destroy()
multi.maxSpd = stop
end
local MaxLoad = nil
function multi:setLoad(n)
MaxLoad = n
end
local busy = false
local lastVal = 0
function multi:getLoad()
if multi.load_updater:isPaused() then multi.load_updater:Resume() return 0 end
local val = math.abs(self.dStepA-self.dStepB)/multi.deltaTarget*100
if val > 100 then return 100 else return val end
if not multi.maxSpd then multi:enableLoadDetection() end
if busy then return lastVal end
local val = nil
if thread.isThread() then
local bench
multi:benchMark(.01):OnBench(function(time,steps)
bench = steps
end)
thread.hold(function()
return bench
end)
bench = bench^1.5
val = math.ceil((1-(bench/(multi.maxSpd/1.5)))*100)
else
busy = true
local bench
multi:benchMark(.01):OnBench(function(time,steps)
bench = steps
end)
while not bench do
multi:uManager()
end
bench = bench^1.5
val = math.ceil((1-(bench/(multi.maxSpd/1.5)))*100)
busy = false
end
if val<0 then val = 0 end
if val > 100 then val = 100 end
lastVal = val
return val
end
function multi:setDomainName(name)
self[name]={}
@ -282,9 +331,6 @@ function multi.AlignTable(tab)
return table.concat(str,"\n")
end
function multi:getTasksDetails(t)
if not multi.load_updater then
multi:enableLoadDetection()
end
if t == "string" or not t then
str = {
{"Type","Uptime","Priority","TID"}
@ -296,7 +342,6 @@ function multi:getTasksDetails(t)
end
table.insert(str,{v.Type:sub(1,1):upper()..v.Type:sub(2,-1)..name,multi.Round(os.clock()-v.creationTime,3),self.PriorityResolve[v.Priority],i})
end
local s = multi.AlignTable(str)
dat = ""
for i=1,#multi.scheduler.Threads do
@ -563,7 +608,7 @@ function multi:newProcessor(file)
end
end)
c.l.link = c
c.l.Type = "process"
c.l.Type = "processor"
function c:getController()
return c.l
end
@ -585,15 +630,19 @@ function multi:newProcessor(file)
return self
end
function c:Remove()
if self.Type == "process" then
self:__Destroy()
self.l:Destroy()
else
self:__Destroy()
end
end
if file then
self.Cself=c
loadstring('local process=multi.Cself '..io.open(file,'rb'):read('*all'))()
end
c.__Destroy = self.Destroy
c.Destroy = c.Remove
-- c.__Destroy = self.Destroy
-- c.Destroy = c.Remove
self:create(c)
--~ c:IngoreObject()
return c
@ -813,7 +862,7 @@ function multi.nextStep(func)
if not next then
next = {func}
else
next[#self.next+1] = func
next[#next+1] = func
end
end
multi.OnPreLoad=multi:newConnection()
@ -1808,12 +1857,12 @@ function multi:mainloop(settings)
local solid
local sRef
while mainloopActive do
--print(mainloopActive)
if ncount ~= 0 then
for i = 1, ncount do
next[i]()
if next then
local DD = table.remove(next,1)
while DD do
DD()
DD = table.remove(next,1)
end
ncount = 0
end
if priority == 1 then
for _D=#Loop,1,-1 do
@ -2313,7 +2362,7 @@ function multi:newFromString(str)
end
return self
elseif t=="process" then
local temp=multi:newProcess()
local temp=multi:newProcessor()
local objs=handle:getBlock("n",4)
for i=1,objs do
temp:newFromString(handle:getBlock("s",(handle:getBlock("n",4))))
@ -2377,22 +2426,4 @@ end
function multi:setDefualtStateFlag(opt)
--
end
function multi:enableLoadDetection()
if multi.load_updater then return end
multi.dStepA = 0
multi.dStepB = 0
multi.dSwap = 0
multi.deltaTarget = .05
multi.load_updater = multi:newUpdater(2)
multi.load_updater:setName("LoadDetector")
multi.load_updater:OnUpdate(function(self)
if self.Parent.dSwap == 0 then
self.Parent.dStepA = os.clock()
self.Parent.dSwap = 1
else
self.Parent.dSwap = 0
self.Parent.dStepB = os.clock()
end
end)
end
return multi

View File

@ -1,38 +1,51 @@
package.path="?/init.lua;?.lua;"..package.path
multi = require("multi")
local GLOBAL,THREAD = require("multi.integration.lanesManager").init()
nGLOBAL = require("multi.integration.networkManager").init()
local a
function multi:setName(name)
self.Name = name
end
local clock = os.clock
function sleep(n) -- seconds
local t0 = clock()
while clock() - t0 <= n do end
end
master = multi:newMaster{
name = "Main", -- the name of the master
--noBroadCast = true, -- if using the node manager, set this to true to avoid double connections
--~ local GLOBAL,THREAD = require("multi.integration.lanesManager").init()
--~ nGLOBAL = require("multi.integration.networkManager").init()
--~ local a
--~ function multi:setName(name)
--~ self.Name = name
--~ end
--~ local clock = os.clock
--~ function sleep(n) -- seconds
--~ local t0 = clock()
--~ while clock() - t0 <= n do end
--~ end
--~ master = multi:newMaster{
--~ name = "Main", -- the name of the master
--~ --noBroadCast = true, -- if using the node manager, set this to true to avoid double connections
--~ managerDetails = {"192.168.1.4",12345}, -- the details to connect to the node manager (ip,port)
}
master.OnError(function(name,err)
print(name.." has encountered an error: "..err)
end)
local connlist = {}
multi:newThread("NodeUpdater",function()
--~ }
--~ master.OnError(function(name,err)
--~ print(name.." has encountered an error: "..err)
--~ end)
--~ local connlist = {}
--~ multi:newThread("NodeUpdater",function()
--~ while true do
--~ thread.sleep(1)
--~ for i=1,#connlist do
--~ conn = master:execute("TASK_MAN",connlist[i], multi:getTasksDetails())
--~ end
--~ end
--~ end)
--~ master.OnNodeConnected(function(name)
--~ table.insert(connlist,name)
--~ end)
--~ multi.OnError(function(...)
--~ print(...)
--~ end)
--~ for i=1,20 do
--~ multi:newLoop(function()
--~ for i=1,500 do
--~ --
--~ end
--~ end)
--~ end
multi:newThread("Test",function()
while true do
thread.sleep(1)
for i=1,#connlist do
conn = master:execute("TASK_MAN",connlist[i], multi:getTasksDetails())
print(multi:getTasksDetails())
end
end
end)
master.OnNodeConnected(function(name)
table.insert(connlist,name)
end)
multi.OnError(function(...)
print(...)
end)
multi:mainloop{
protect = false