From 03c91d84e329896fde79c08edd630bce5856c548 Mon Sep 17 00:00:00 2001 From: Ryan Date: Sun, 2 Jul 2017 20:00:17 -0400 Subject: [PATCH] Doing tests... Love2d Still unstable with new update --- README.html | 6 +- README.md | 8 +-- multi/init.lua | 13 +++-- multi/integration/lanesManager.lua | 11 ++-- multi/integration/loveManager.lua | 46 ++++++++------- multi/integration/shared/shared.lua | 89 ++++++++++++++++++++++++++--- rockspecs/multi-1.8-2.rockspec | 2 +- rockspecs/multi-1.8-3.rockspec | 2 +- 8 files changed, 126 insertions(+), 51 deletions(-) diff --git a/README.html b/README.html index b5fd3f5..4dd3bb4 100644 --- a/README.html +++ b/README.html @@ -186,7 +186,7 @@

INSTALLING

luarocks install multi

Discord

For real-time assistance with my libraries! A place where you can ask questions and get help with any of my libraries. Also you can request features and stuff there as well.

https://discord.gg/U8UspuA

Planned features/TODO

Known Bugs/Issues

In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P

Usage:

 Add system threads for love2d that works like the lanesManager (loveManager, slight differences).
  •  Improve performance of the library
  •  Improve coroutine based threading scheduling
  •  Add more features to support module creators
  •  Make a framework for eaiser thread task distributing
  •  Fix Error handling on threaded multi objects Non threaded multiobjs will crash your program if they error though! Use multi:newThread() of multi:newSystemThread() if your code can error! Unless you use multi:protect() this however lowers performance!
  •  Add multi:OnError(function(obj,err))
  •  sThread.wrap(obj) May or may not be completed Theory: Allows interaction in one thread to affect it in another. The addition to threaded tables may make this possible!
  •  SystemThreaded Actors — After some tests i figured out a way to make this work… It will work slightly different though. This is due to the actor needing to be splittable…
  •  LoadBalancing for system threads (Once SystemThreaded Actors are done)
  •  Add more integrations
  •  Finish the wiki stuff. (11% done)
  •  Test for unknown bugs
    • Known Bugs/Issues
    In regards to integrations, thread cancellation works slightly different for love2d and lanes. Within love2d I was unable to (To lazy to…) not use the multi library within the thread. A fix for this is to call multi:Stop() when you are done with your threaded code! This may change however if I find a way to work around this. In love2d in order to mimic the GLOBAL table I needed the library to constantly sync tha data… You can use the sThread.waitFor(varname), or sThread.hold(func) methods to sync the globals, to get the value instead of using GLOBAL and this could work. If you want to go this route I suggest setting multi.isRunning=true to prevent the auto runner from doing its thing! This will make the multi manager no longer function, but thats the point :P
    Usage:
    functionend)
 multi:mainloop() -- the main loop of the program, multi:umanager() exists as well to allow integration in other loops Ex: love2d love.update function. More on this binding in the wiki!
 
    The library is modular so you only need to require what you need to. Because of this, the global enviroment is altered
    There are many useful objects that you can use

    Check out the wiki for detailed usage, but here are the objects:
      
-
    • Process#
    • Queuer#
    • Alarm
    • Loop
    • Event
    • Step
    • Range
    • TStep
    • TLoop
    • Condition
    • Connection
    • Timer
    • Updater
    • Thread*
    • Trigger
    • Task
    • Job
    • Function
    • Watcher

      Note: Both a process and queue act like the multi namespace, but allows for some cool things. Because they use the other objects an example on them will be done last

      *Uses the built in coroutine features of lua, these have an interesting interaction with the other means of multi-tasking

      Triggers are kind of useless after the creation of the Connection

      Watchers have no real purpose as well I made it just because.
    Examples of each object being used
    We already showed alarms in action so lets move on to a Loop object
    Throughout these examples I am going to do some strange things in order to show other features of the library!
    LOOPS
    Examples of each object being used
    We already showed alarms in action so lets move on to a Loop object
    Throughout these examples I am going to do some strange things in order to show other features of the library!
    LOOPS
    3):OnBench(3,"All Threads: ")
 end)
 multi:mainloop()
-
    Using multi:newSystemThreadedQueue()
    Quick Note: queues shared across multiple objects will be pulling from the same “queue” keep this in mind when coding! Also the queue respects direction a push on the thread side cannot be popped on the thread side… Same goes for the mainthread!
    Using multi:newSystemThreadedQueue()
    Quick Note: queues shared across multiple objects will be pulling from the same “queue” keep this in mind when coding! Also the queue respects direction a push on the thread side cannot be popped on the thread side… Same goes for the mainthread!
     Turns out i was wrong about this…
    
+Quick Note: queues shared across multiple objects will be pulling from the same "queue" keep this in mind when coding! ~~Also the queue respects direction a push on the thread side cannot be popped on the thread side... Same goes for the mainthread!
    ~~ Turns out i was wrong about this...
 ```lua
 -- in love2d, this file will be in the same example folder as before, but is named main2.lua
 require("core.Library")
diff --git a/multi/init.lua b/multi/init.lua
index 3bd3e8d..42f8b74 100644
--- a/multi/init.lua
+++ b/multi/init.lua
@@ -45,8 +45,8 @@ function print(...)
 	end
 end
 multi = {}
-multi.Version="1.8.3"
-multi._VERSION="1.8.3"
+multi.Version="1.8.4"
+multi._VERSION="1.8.4"
 multi.stage='stable'
 multi.__index = multi
 multi.Mainloop={}
@@ -980,6 +980,7 @@ function multi:mainloop()
 	end
 end
 function multi:protectedMainloop()
+	multi:protect()
 	if not multi.isRunning then
 		multi.isRunning=true
 		for i=1,#self.Tasks do
@@ -994,6 +995,7 @@ function multi:protectedMainloop()
 	end
 end
 function multi:unprotectedMainloop()
+	multi:unProtect()
 	if not multi.isRunning then
 		multi.isRunning=true
 		for i=1,#self.Tasks do
@@ -1018,6 +1020,7 @@ function multi:unprotectedMainloop()
 	end
 end
 function multi:prioritizedMainloop1()
+	multi:enablePriority()
 	if not multi.isRunning then
 		multi.isRunning=true
 		for i=1,#self.Tasks do
@@ -1049,6 +1052,7 @@ function multi:prioritizedMainloop1()
 	end
 end
 function multi:prioritizedMainloop2()
+	multi:enablePriority2()
 	if not multi.isRunning then
 		multi.isRunning=true
 		for i=1,#self.Tasks do
@@ -1593,7 +1597,7 @@ function multi:newThread(name,func)
 end
 multi:setDomainName("Threads")
 multi:setDomainName("Globals")
-multi.scheduler=multi:newUpdater()
+multi.scheduler=multi:newLoop()
 multi.scheduler.Type="scheduler"
 function multi.scheduler:setStep(n)
 	self.skip=tonumber(n) or 24
@@ -1602,7 +1606,7 @@ multi.scheduler.skip=0
 multi.scheduler.counter=0
 multi.scheduler.Threads=multi:linkDomain("Threads")
 multi.scheduler.Globals=multi:linkDomain("Globals")
-multi.scheduler:OnUpdate(function(self)
+multi.scheduler:OnLoop(function(self)
 	self.counter=self.counter+1
 	for i=#self.Threads,1,-1 do
 		ret={}
@@ -1656,7 +1660,6 @@ multi.scheduler:OnUpdate(function(self)
 		end
 	end
 end)
-multi.scheduler:setStep()
 multi.scheduler:Pause()
 multi.OnError=multi:newConnection()
 function multi:newThreadedAlarm(name,set)
diff --git a/multi/integration/lanesManager.lua b/multi/integration/lanesManager.lua
index 49a81f8..9c0e60b 100644
--- a/multi/integration/lanesManager.lua
+++ b/multi/integration/lanesManager.lua
@@ -125,12 +125,9 @@ function multi:newSystemThread(name,func)
 	end)
     return c
 end
-print("Intergrated Lanes!")
-multi.intergration={} -- for module creators
-multi.intergration.GLOBAL=GLOBAL
-multi.intergration.THREAD=THREAD
-multi.intergration.lanes={}
-multi.intergration.lanes.GLOBAL=GLOBAL -- for module creators
-multi.intergration.lanes.THREAD=THREAD -- for module creators
+print("Integrated Lanes!")
+multi.integration={} -- for module creators
+multi.integration.GLOBAL=GLOBAL
+multi.integration.THREAD=THREAD
 require("multi.integration.shared.shared")
 return {init=function() return GLOBAL,THREAD end}
diff --git a/multi/integration/loveManager.lua b/multi/integration/loveManager.lua
index 7422f38..e357732 100644
--- a/multi/integration/loveManager.lua
+++ b/multi/integration/loveManager.lua
@@ -2,14 +2,14 @@ require("multi.compat.love2d")
 function multi:canSystemThread()
 	return true
 end
-multi.intergration={}
-multi.intergration.love2d={}
-multi.intergration.love2d.ThreadBase=[[
+multi.integration={}
+multi.integration.love2d={}
+multi.integration.love2d.ThreadBase=[[
 __THREADNAME__=({...})[1]
 require("love.filesystem")
 require("love.system")
 require("love.timer")
-require("multi.all")
+require("multi")
 GLOBAL={}
 setmetatable(GLOBAL,{
 	__index=function(t,k)
@@ -79,7 +79,7 @@ function resolveType(tp,d)
 	elseif tp=="bool" then
 		return (d=="true")
 	elseif tp=="function" then
-		return loadDump(d)
+		return loadDump("[==["..d.."]==]")
 	elseif tp=="table" then
 		return loadstring("return "..d)()
 	elseif tp=="nil" then
@@ -91,11 +91,11 @@ end
 function resolveData(v)
 	local data=""
 	if type(v)=="table" then
-		data=ToStr(v)
+		return ToStr(v)
 	elseif type(v)=="function" then
-		data=dump(v)
+		return dump(v)
 	elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then
-		data=tostring(v)
+		return tostring(v)
 	end
 	return data
 end
@@ -191,7 +191,7 @@ setmetatable(GLOBAL,{
 	end,
 })
 THREAD={} -- Allow main thread to interact with these objects as well
-multi.intergration.love2d.mainChannel=love.thread.getChannel("__MainChan__")
+multi.integration.love2d.mainChannel=love.thread.getChannel("__MainChan__")
 function ToStr(val, name, skipnewlines, depth)
     skipnewlines = skipnewlines or false
     depth = depth or 0
@@ -228,7 +228,7 @@ function resolveType(tp,d)
 	elseif tp=="bool" then
 		return (d=="true")
 	elseif tp=="function" then
-		return loadDump(d)
+		return loadDump("[==["..d.."]==]")
 	elseif tp=="table" then
 		return loadstring("return "..d)()
 	elseif tp=="nil" then
@@ -240,11 +240,11 @@ end
 function resolveData(v)
 	local data=""
 	if type(v)=="table" then
-		data=ToStr(v)
+		return ToStr(v)
 	elseif type(v)=="function" then
-		data=dump(v)
+		return dump(v)
 	elseif type(v)=="string" or type(v)=="number" or type(v)=="bool" or type(v)=="nil" then
-		data=tostring(v)
+		return tostring(v)
 	end
 	return data
 end
@@ -274,10 +274,10 @@ function multi:newSystemThread(name,func) -- the main method
     local c={}
     c.name=name
 	c.ID=c.name..""
-    c.thread=love.thread.newThread(multi.intergration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func)))
+    c.thread=love.thread.newThread(multi.integration.love2d.ThreadBase:gsub("INSERT_USER_CODE",dump(func)))
 	c.thread:start(c.ID)
 	function c:kill()
-		multi.intergration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__"
+		multi.integration.GLOBAL["__DIEPLZ"..self.ID.."__"]="__DIEPLZ"..self.ID.."__"
 	end
 	return c
 end
@@ -310,11 +310,11 @@ function THREAD.hold(n)
 	multi.OBJ_REF:Resume()
 end
 __channels__={}
-multi.intergration.GLOBAL=GLOBAL
-multi.intergration.THREAD=THREAD
+multi.integration.GLOBAL=GLOBAL
+multi.integration.THREAD=THREAD
 updater=multi:newUpdater()
 updater:OnUpdate(function(self)
-	local data=multi.intergration.love2d.mainChannel:pop()
+	local data=multi.integration.love2d.mainChannel:pop()
 	while data do
 		--print("MAIN:",data)
 		if type(data)=="string" then
@@ -342,19 +342,21 @@ updater:OnUpdate(function(self)
 		else
 			__proxy__[name]=data
 		end
-		data=multi.intergration.love2d.mainChannel:pop()
+		data=multi.integration.love2d.mainChannel:pop()
 	end
 end)
 require("multi.integration.shared.shared")
-print("Intergrated Love2d!")
+print("Integrated Love2d!")
 return {
 	init=function(t)
 		if t then
 			if t.threadNamespace then
-				multi.intergration.love2d.ThreadBase:gsub("sThread",t.threadNamespace)
+				multi.integration.THREADNAME=t.threadNamespace
+				multi.integration.love2d.ThreadBase:gsub("sThread",t.threadNamespace)
 			end
 			if t.globalNamespace then
-				multi.intergration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace)
+				multi.integration.GLOBALNAME=t.globalNamespace
+				multi.integration.love2d.ThreadBase:gsub("GLOBAL",t.globalNamespace)
 			end
 		end
 		return GLOBAL,THREAD
diff --git a/multi/integration/shared/shared.lua b/multi/integration/shared/shared.lua
index 023b38f..b393612 100644
--- a/multi/integration/shared/shared.lua
+++ b/multi/integration/shared/shared.lua
@@ -29,10 +29,16 @@ function multi:newSystemThreadedQueue(name) -- in love2d this will spawn a chann
 			function c:init() -- create an init function so we can mimic on bith love2d and lanes
 				self.chan=love.thread.getChannel(self.name) -- create channel by the name self.name
 				function self:push(v) -- push to the channel
-					self.chan:push(v)
+					self.chan:push({type(v),resolveData(v)})
 				end
 				function self:pop() -- pop from the channel
-					return self.chan:pop()
+					local tab=self.chan:pop()
+					if not tab then return end
+					return resolveType(tab[1],tab[2])
+				end
+				function self:peek()
+					local tp,d=unpack{self.chan:peek()}
+					return resolveType(tp,d)
 				end
 				GLOBAL[self.name]=self -- send the object to the thread through the global interface
 				return self -- return the object
@@ -49,20 +55,23 @@ function multi:newSystemThreadedQueue(name) -- in love2d this will spawn a chann
 		function c:pop() -- pop the queue
 			return ({self.linda:receive(0,"Q")})[2]
 		end
+		function c:peek()
+			return self.linda:get("Q")
+		end
 		function c:init() -- mimic the feature that love2d requires, so code can be consistent
 			return self
 		end
-		multi.intergration.GLOBAL[name]=c -- send the object to the thread through the global interface
+		multi.integration.GLOBAL[name]=c -- send the object to the thread through the global interface
 	end
 	return c
 end
 function multi:systemThreadedBenchmark(n,p)
 	n=n or 1
-	local cores=multi.intergration.THREAD.getCores()
+	local cores=multi.integration.THREAD.getCores()
 	local queue=multi:newSystemThreadedQueue("QUEUE")
-	multi.intergration.GLOBAL["__SYSTEMBENCHMARK__"]=n
-	local sThread=multi.intergration.THREAD
-	local GLOBAL=multi.intergration.GLOBAL
+	multi.integration.GLOBAL["__SYSTEMBENCHMARK__"]=n
+	local sThread=multi.integration.THREAD
+	local GLOBAL=multi.integration.GLOBAL
 	for i=1,cores do
 		multi:newSystemThread("STHREAD_BENCH",function()
 			require("multi")
@@ -170,7 +179,71 @@ function multi:newSystemThreadedTable(name)
 			})
 			return self
 		end
-		multi.intergration.GLOBAL[name]=c -- send the object to the thread through the global interface
+		multi.integration.GLOBAL[name]=c -- send the object to the thread through the global interface
 	end
 	return c
 end
+function multi:newSystemThreadedJobQueue(numOfCores)
+	local c={}
+	c.jobnum=1
+	c.cores=numOfCores or multi.integration.THREAD.getCores()
+	c.queueIN=multi:newSystemThreadedQueue("THREADED_JQ"):init()
+	c.queueOUT=multi:newSystemThreadedQueue("THREADED_JQO"):init()
+	c.REG=multi:newSystemThreadedTable("THREADED_JQ_F_REG"):init()
+	-- registerJob(name,func)
+	-- pushJob(...)
+	function c:registerJob(name,func)
+		self.REG[name]=func
+	end
+	function c:pushJob(name,...)
+		self.queueOUT:push({self.jobnum,name,...})
+		self.jobnum=self.jobnum+1
+	end
+	local GLOBAL=multi.integration.GLOBAL -- set up locals incase we are using lanes
+	local sThread=multi.integration.THREAD -- set up locals incase we are using lanes
+	GLOBAL["__JQ_COUNT__"]=c.cores
+	for i=1,c.cores do
+		multi:newSystemThread("System Threaded Job Queue Worker Thread #"..i,function()
+			require("multi")
+			__sleep__=.001
+			if love then -- lets make sure we don't reference upvalues if using love2d
+				GLOBAL=_G.GLOBAL
+				sThread=_G.sThread
+				__sleep__=.1
+			end
+			JQI=sThread.waitFor("THREADED_JQO"):init() -- Grab it
+			JQO=sThread.waitFor("THREADED_JQ"):init() -- Grab it
+			FGLOBAL=sThread.waitFor("THREADED_JQ_F_REG"):init() -- Grab it
+			sThread.sleep(.1) -- lets wait for things to work out
+			setmetatable(_G,{
+				__index=FGLOBAL
+			})
+			GLOBAL["THREADED_JQ"]=nil -- remove it
+			GLOBAL["THREADED_JQO"]=nil -- remove it
+			GLOBAL["THREADED_JQ_F_REG"]=nil -- remove it
+			multi:newLoop(function()
+				sThread.sleep(__sleep__) -- lets allow cpu time for other processes on our system!
+				local job=JQI:pop()
+				if job then
+					local ID=table.remove(job,1) -- return and remove
+					local name=table.remove(job,1) -- return and remove
+					local ret={FGLOBAL:waitFor(name)(unpack(job))} -- unpack the rest
+					JQO:push({ID,ret})
+				end
+			end)
+			multi:mainloop()
+		end)
+	end
+	c.OnJobCompleted=multi:newConnection()
+	c.updater=multi:newLoop(function(self)
+		local data=self.link.queueIN:pop()
+		while data do
+			if data then
+				self.link.OnJobCompleted:Fire(unpack(data))
+			end
+			data=self.link.queueIN:pop()
+		end
+	end)
+	c.updater.link=c
+	return c
+end
diff --git a/rockspecs/multi-1.8-2.rockspec b/rockspecs/multi-1.8-2.rockspec
index 2125f12..6ffbdf9 100644
--- a/rockspecs/multi-1.8-2.rockspec
+++ b/rockspecs/multi-1.8-2.rockspec
@@ -22,7 +22,7 @@ build = {
       ["multi.init"] = "multi/init.lua",
       ["multi.all"] = "multi/all.lua",
       ["multi.compat.backwards[1,5,0]"] = "multi/compat/backwards[1,5,0].lua",
-      ["multi.compat"] = "multi/compat/love2d.lua",
+      ["multi.compat.love2d"] = "multi/compat/love2d.lua",
       ["multi.integration.lanesManager"] = "multi/integration/lanesManager.lua",
       ["multi.integration.loveManager"] = "multi/integration/loveManager.lua",
       ["multi.integration.shared.shared"] = "multi/integration/shared/shared.lua"
diff --git a/rockspecs/multi-1.8-3.rockspec b/rockspecs/multi-1.8-3.rockspec
index 88088ad..e3e152f 100644
--- a/rockspecs/multi-1.8-3.rockspec
+++ b/rockspecs/multi-1.8-3.rockspec
@@ -22,7 +22,7 @@ build = {
       ["multi.init"] = "multi/init.lua",
       ["multi.all"] = "multi/all.lua",
       ["multi.compat.backwards[1,5,0]"] = "multi/compat/backwards[1,5,0].lua",
-      ["multi.compat"] = "multi/compat/love2d.lua",
+      ["multi.compat.love2d"] = "multi/compat/love2d.lua",
       ["multi.integration.lanesManager"] = "multi/integration/lanesManager.lua",
       ["multi.integration.loveManager"] = "multi/integration/loveManager.lua",
       ["multi.integration.shared.shared"] = "multi/integration/shared/shared.lua"