GraphicsCanvas - How to hide it properly?

Archives Forums/Linux Discussion/GraphicsCanvas - How to hide it properly?

Grisu

(Posted 2012) [#1]

Hi!

I'm using a canvas in order to display a logo and a spectrum analyser.

The issue under Linux: The canvas will still receive updates via eventhooks even if the MainWindow is hidden behind other windows?!

Example:

I could call it a feature. But I'd rather like to get rid of this... :(

Any work around present?

Grisu

Derron

(Posted 2012) [#2]

double post

Last edited 2012

Derron

(Posted 2012) [#3]

couldn't you deactivate the event listener on "app hidden" ?.

How did you connect your event listener?
I do not have problems with the max2d-canvas on background windows...

Type TGUIScrollableCanvas
	Field backgroundPanel:TGadget			'parent of canvas
	Field canvas:TGadget
	Field realDimension:TPoint	= TPoint.Create(0,0)
	Field showDimension:TPoint	= TPoint.Create(0,0)
	Field showOffset:TPoint		= TPoint.Create(0,0)
	Field scrollBarX:TGadget, scrollBarY:TGadget
	Global List:TList = CreateList() 'all scrollable canvas
	Global scrollbarSize:int = 15

	Function Create:TGUIScrollableCanvas(realW:int, realH:int, parent:TGadget, padding:int=0)
		local obj:TGUIScrollableCanvas = new TGUIScrollableCanvas
		obj.backgroundPanel = parent

		obj.showDimension.SetX(ClientWidth(obj.backgroundPanel)-scrollbarSize-2.5*padding)
		obj.showDimension.SetY(ClientHeight(obj.backgroundPanel)-scrollbarSize-2.5*padding)

		obj.realDimension.SetXY( realW, realH)

		'canvas is added to the scrollable main panel
		obj.canvas = CreateCanvas(padding,padding,obj.showDimension.getX(),obj.showDimension.getY(),obj.backgroundPanel)
		'so it resizes -> pin to the corners
		SetGadgetLayout( obj.Canvas, EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED )

		obj.scrollBarX = CreateSlider(padding,ClientHeight(obj.backgroundPanel)-scrollbarSize-1.5*padding,ClientWidth(obj.backgroundPanel)-scrollbarSize-2.5*padding,scrollbarSize,obj.backgroundPanel,SLIDER_HORIZONTAL)
		obj.scrollBarY = CreateSlider(ClientWidth(obj.backgroundPanel)-scrollbarSize-1.5*padding,padding,scrollbarSize,ClientHeight(obj.backgroundPanel)-scrollbarSize-2.5*padding,obj.backgroundPanel,SLIDER_VERTICAL)
		obj.scrollBarX.SetLayout(EDGE_ALIGNED,EDGE_ALIGNED,EDGE_CENTERED,EDGE_ALIGNED)
		obj.scrollBarY.SetLayout(EDGE_CENTERED,EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED)

		obj.List.AddLast(obj)
		return obj
	End Function

	Method New()
		AddHook EmitEventHook,eventhook,Self
	End Method

	Method Free()
		RemoveHook EmitEventHook,eventhook
		GCCollect()
	End Method

	Method New()
		AddHook EmitEventHook,eventhook,Self
	End Method

	Function eventhook:Object(id:Int,data:Object,context:Object)
		If TGUIScrollableCanvas(context)
			TGUIScrollableCanvas(context).processEvent(TEvent(data))
			Return data
		EndIf
	EndFunction

	Function ProcessEvent:TEvent(event:TEvent)
		local instance:TGUIScrollableCanvas = null
		For local obj:TGUIScrollableCanvas = eachin TGUIScrollableCanvas.list
			if Event.Source = obj.canvas then instance = obj;exit
			if Event.Source = obj.scrollbarX then instance = obj;exit
			if Event.Source = obj.scrollbarY then instance = obj;exit
		Next
		if not instance then return Event

		'Canvas event
		Select Event.id
			Case EVENT_GADGETPAINT
				'do the paint things ... eg custom canvas.paint-method
				'instance.Refresh()

				 'we handled the canvas paint - do not pass through
				Return Null
			Case EVENT_GADGETACTION
				Select Event.Source
					'Case instance.scrollBarX
					'	instance.showOffset.setX(-Event.data)
					'	instance.Refresh()
				End Select
		End Select
		'Passback event (pass through)
		Return Event
	End Function
...

the above code is an excerpt from a custom class. - which uses the canvas and is drawn here on linux mint 64 without problems (using fltk).

bye
Ron

edit: code->codebox

Last edited 2012

Derron

(Posted 2012) [#4]

Dunno why the above is posted twice... maybe one is not allowed to "edit", alt+left, edit again...

Ok here is some test code... it redraws the background although if the app windows is not running in front.

SuperStrict

Framework MaxGUI.Drivers
'Import bah.gtkmaxgui
Import BRL.GLMax2D
Import BRL.EventQueue
Import BRL.retro
Import BRL.timer


Global App:TApp	= New TApp
'run the app - and loop etc.
App.Run



Type TApp
	Global instance:TApp		= Null			'singleton principle
	'gfx/gui
	Global guiWindow:TGadget	= Null
	Global guiMainPanel:TGadget	= null
	Global guiScrollableCanvas:TGUIScrollableCanvas = null
	Global guiFileMenu:TGadget	= Null
	Global EVENT_FASTKEYHIT:Int	= AllocUserEventId("fastkeyhit") 'faster key response

	Const MENU_OPEN:Int		= 101
	Const MENU_EXIT:Int		= 102

	Method New()
		Self.instance = Self
	End Method

	Function GetInstance:TApp()
		If TApp.instance Then Return TApp.instance
		Return New TApp
	End Function

	Method Run:TApp()
		guiWindow = CreateWindow( "Test", 100, 100, 800, 550, Null, WINDOW_MENU|WINDOW_TITLEBAR|WINDOW_RESIZABLE|WINDOW_STATUS|WINDOW_CLIENTCOORDS )

		'panel for main+right, pin at all corners
		guiMainPanel =  CreatePanel(0,0,ClientWidth(guiWindow),ClientHeight(guiWindow),guiWindow,PANEL_GROUP, "TestPanel")
		guiMainPanel.SetLayout( EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED )

		'main area
		guiScrollableCanvas	= TGUIScrollableCanvas.Create(768,768, guiMainPanel, 0)
		guiScrollableCanvas.Redraw()

		'FileMenu
		guiFileMenu = CreateMenu("&Datei",0,WindowMenu(guiWindow))
		CreateMenu( "&Open Something"	, MENU_OPEN	,guiFileMenu	, KEY_V, MODIFIER_COMMAND )
		CreateMenu( ""						, 0			,guiFileMenu )
		CreateMenu( "&Exit"				, MENU_EXIT	,guiFileMenu	, KEY_F4, MODIFIER_COMMAND )
		'add menus
		UpdateWindowMenu(guiWindow)

		'bug test: redraw app although if in background (once every second)
		'timed canvas update
		local myTimer:TTimer = CreateTimer:TTimer( 1 )

		While True
'			ActivateGadget(guiCanvas)
			WaitEvent
			Select EventID()
					'testcase - timed canvas update (for background-draw-test)
					Case EVENT_TIMERTICK
						print "redraw" + Millisecs()
						guiScrollableCanvas.Redraw()

					Case EVENT_MENUACTION
						Select EventData()
							Case MENU_EXIT
									End
							Case MENU_OPEN
									'open file
									'in case of changes:
									'guiScrollableCanvas.Redraw()
						End Select
					'checking both events -> faster response on "keep pressed"
					Case EVENT_KEYREPEAT
						Self.HandleNavigatorKeys(CurrentEvent)
					Case EVENT_KEYDOWN
						Self.HandleNavigatorKeys(CurrentEvent)
						guiScrollableCanvas.Redraw()
					Case EVENT_GADGETACTION
						'check EventSource() against other interactable objects (eg lists)
						'if EventSource() = self.guiSpritesList
							'...
							'guiScrollableCanvas.Redraw()
						'endif
					Case EVENT_WINDOWCLOSE
						End
			End Select
		Wend

		Return Self
	End Method


	Method HandleNavigatorKeys(event:TEvent)
		If event.data = KEY_DOWN Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_DOWN))
		If event.data = KEY_UP Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_UP))
		If event.data = KEY_LEFT Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_LEFT))
		If event.data = KEY_RIGHT Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_RIGHT))
	End Method
End Type





Type TGUIScrollableCanvas
	Field backgroundPanel:TGadget			'parent of canvas
	Field canvas:TGadget
	Field realDimension:TPoint	= TPoint.Create(0,0)
	Field showDimension:TPoint	= TPoint.Create(0,0)
	Field showOffset:TPoint		= TPoint.Create(0,0)
	Field scrollBarX:TGadget, scrollBarY:TGadget
	Global List:TList = CreateList()
	Global scrollbarSize:int = 15

	Function Create:TGUIScrollableCanvas(realW:int, realH:int, parent:TGadget, padding:int=0)
		local obj:TGUIScrollableCanvas = new TGUIScrollableCanvas
		obj.backgroundPanel = parent

		obj.showDimension.SetX(ClientWidth(obj.backgroundPanel)-scrollbarSize-2.5*padding)
		obj.showDimension.SetY(ClientHeight(obj.backgroundPanel)-scrollbarSize-2.5*padding)

		obj.realDimension.SetXY( realW, realH)

		'canvas is added to the scrollable main panel
		obj.canvas = CreateCanvas(padding,padding,obj.showDimension.getX(),obj.showDimension.getY(),obj.backgroundPanel)
		'so it resizes -> pin to the corners
		SetGadgetLayout( obj.Canvas, EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED )

		obj.scrollBarX = CreateSlider(padding,ClientHeight(obj.backgroundPanel)-scrollbarSize-1.5*padding,ClientWidth(obj.backgroundPanel)-scrollbarSize-2.5*padding,scrollbarSize,obj.backgroundPanel,SLIDER_HORIZONTAL)
		obj.scrollBarY = CreateSlider(ClientWidth(obj.backgroundPanel)-scrollbarSize-1.5*padding,padding,scrollbarSize,ClientHeight(obj.backgroundPanel)-scrollbarSize-2.5*padding,obj.backgroundPanel,SLIDER_VERTICAL)
		obj.scrollBarX.SetLayout(EDGE_ALIGNED,EDGE_ALIGNED,EDGE_CENTERED,EDGE_ALIGNED)
		obj.scrollBarY.SetLayout(EDGE_CENTERED,EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED)

		obj.List.AddLast(obj)
		return obj
	End Function

	Method New()
		AddHook EmitEventHook,eventhook,Self
	End Method

	Method Free()
		RemoveHook EmitEventHook,eventhook
		GCCollect()
	End Method

	Function eventhook:Object(id:Int,data:Object,context:Object)
		If TGUIScrollableCanvas(context)
			TGUIScrollableCanvas(context).processEvent(TEvent(data))
			Return data
		EndIf
	EndFunction

	Function ProcessEvent:TEvent(event:TEvent)
		local instance:TGUIScrollableCanvas = null
		For local obj:TGUIScrollableCanvas = eachin TGUIScrollableCanvas.list
			if Event.Source = obj.canvas then instance = obj;exit
			if Event.Source = obj.scrollbarX then instance = obj;exit
			if Event.Source = obj.scrollbarY then instance = obj;exit
		Next
		if not instance then return Event

		'Canvas event
		Select Event.id
			Case EVENT_GADGETPAINT
				instance.Refresh()

				 'we handled the canvas paint - do not pass through
				Return Null
			Case EVENT_GADGETACTION
				Select Event.Source
					Case instance.scrollBarX
						instance.showOffset.setX(-Event.data)
						instance.Refresh()
					Case instance.scrollBarY
						instance.showOffset.setY(-Event.data)
						instance.Refresh()
				End Select
		End Select
		'Passback event (pass through)
		Return Event
	End Function

	Method Refresh()
		'adjust dimensions
		self.showDimension.SetXY(GadgetWidth(self.canvas), GadgetHeight(self.canvas))
		'adjust scrollbar values
		self.scrollBarX.SetRange(GadgetWidth(self.Canvas), self.realDimension.getX())
		self.scrollBarY.SetRange(GadgetHeight(self.Canvas), self.realDimension.getY())


		self.Draw()
	End Method

	Method Redraw()
		RedrawGadget( self.canvas )
	End Method

	Method Draw()

		SetGraphics CanvasGraphics(self.canvas)
		SetBlend AlphaBlend
		SetViewport( 0,0,self.showDimension.getX(),self.showDimension.getY() )

		SetClsColor 255,255,255
		Cls
		Self.DrawCanvasBackground()

		SetColor 0,0,0
		DrawText("display: "+ showDimension.getIntX()+"x"+showDimension.getIntY()+ "  real: "+realDimension.getIntX()+"x"+realDimension.getIntY(), 0,showDimension.getIntY()-15)
		SetColor 255,255,255

		'trigger draw event: run all listeners now (instead later)
		'following is a custom Event-System - normally you should use the default TEvent-approach
		'EventManager.triggerEvent( TEventSimple.Create("gui.canvas.onDraw", TData.Create().AddNumber("offsetX", self.showOffset.GetX()).AddNumber("offsetY", self.showOffset.GetY()), self.canvas ) )
		Flip
	End Method



	Method DrawCanvasBackground()
		Local i:Int = 0

		'darker background if canvas is bigger than real image
		if self.realDimension.getX() < self.showDimension.getX() or ..
		   self.realDimension.getY() < self.showDimension.getY()

			local bgColor:int = 150
			SetColor bgColor,bgColor,bgColor
			DrawRect(0,0,self.showDimension.getX(),self.showDimension.getY())
			SetColor 255,255,255
			DrawRect(0,0,self.realDimension.getX(),self.realDimension.getY())
			'shadow
			local steps:int = 4
			local color:int = 50	'start color
			local colorStep:int = (bgColor-color)/steps
			for local i:int = 0 to steps-1
				SetColor color,color,color
				'horizontal width +1 so it connects with vertical ones
				DrawRect(i,realDimension.getY()+i,realDimension.getX()+1,1)
				DrawRect(realDimension.getX()+i,i,1,realDimension.getY())
				color :+ colorStep
			Next
		endif

		SetColor 230,230,230
		local dimension:int	= 20
		local displaceX:int	= 0
		local startX:int	= self.showOffset.getX()
		local startY:int	= self.showOffset.getY()
		local endX:int		= Min(self.realDimension.getX(), self.showDimension.getX())
		local endY:int		= Min(self.realDimension.getY(), self.showDimension.getY())
		For local y:int = startY to endY
			For local x:int = startX to endX
				local width:int = Max( 0, Min(dimension, endX-(x+displaceX)) )
				DrawRect(x+displaceX,y,width,Min(endY-y,dimension))
				x :+ dimension*2 'skip one each time
			Next
			displaceX = dimension - displaceX
			y :+ dimension 'next line
		Next
	End Method

End Type






'helper
Type TPoint {_exposeToLua="selected"}
	Field x:Float
	Field y:Float

	Function Create:TPoint(_x:Float=0.0,_y:Float=0.0)
		Local tmpObj:TPoint = New TPoint
		tmpObj.SetX(_x)
		tmpObj.SetY(_y)
		Return tmpObj
	End Function

	Function CreateFromPos:TPoint(pos:TPoint)
		return TPoint.Create(pos.x,pos.y)
	End Function

	Method GetIntX:int() {_exposeToLua}
		return floor(self.x)
	End Method

	Method GetIntY:int() {_exposeToLua}
		return floor(self.y)
	End Method


	Method GetX:float() {_exposeToLua}
		return self.x
	End Method

	Method GetY:float() {_exposeToLua}
		return self.y
	End Method

	Method SetX(_x:Float)
		Self.x = _x
	End Method

	Method SetY(_y:Float)
		Self.y = _y
	End Method

	Method SetXY(_x:Float, _y:Float)
		Self.SetX(_x)
		Self.SetY(_y)
	End Method

	Method SetPos(otherPos:TPoint)
		Self.SetX(otherPos.x)
		Self.SetY(otherPos.y)
	End Method

	Method MoveXY( _x:float, _y:float )
		Self.x:+ _x
		Self.y:+ _y
	End Method

	Method isSame:int(otherPos:TPoint, round:int=0) {_exposeToLua}
		if round
			return abs(self.x -otherPos.x)<1.0 AND abs(self.y -otherPos.y) < 1.0
		else
			return self.x = otherPos.x AND self.y = otherPos.y
		endif
	End Method


	Function SwitchPos(Pos:TPoint Var, otherPos:TPoint Var)
		Local oldx:Float, oldy:Float
		oldx = Pos.x
		oldy = Pos.y
		Pos.x = otherpos.x
		Pos.y = otherpos.y
		otherpos.x = oldx
		otherpos.y = oldy
	End Function

End Type

edit: code->codebox

Last edited 2012

Grisu

(Posted 2012) [#5]

Thanks for looking into this.

The BMX IDE keeps crashing for me. So I couldn't do much with BMX-wise under Linux. :(

I use a much simpler approach to redraw the canvas:

Function Redraw_RadioStationCanvas:Object(iId:Int, tData:Object, tContext:Object)
	    Local Event:TEvent=TEvent(tData)
		If event <> Null And RadioStationCanvas <> Null Then 
			If (Event.Source = RadioStationCanvas) And (Event.ID = EVENT_GADGETPAINT)
                 SetGraphics CanvasGraphics( RadioStationCanvas )
			Flip 0		
			Return Null
			EndIf	
		EndIf		 
	Return tData

Your source code #2 shows exactly the same issue that my code has. :(

While the BMX IDE worked, I compiled a full version of my radio player to show this and other issues (hidden panels/GadgetRedraw issues etc).

Download #3 (~11 MB):
http://www.mediafire.com/?aks4rel8y239pru

Last edited 2012

Last edited 2012

Brucey

(Posted 2012) [#6]

The BMX IDE keeps crashing for me.

Oh dear...

You can compile on the command-line if you know the right parameters to pass to bmk.

Derron

(Posted 2012) [#7]

Your download links to "icontest" (source) which runs flawless.
->stays in background and nothing is drawn
-> no icon is shown (just gray background).

Are you running the latest blitzmax ?
What OS?

bye
Ron

edit: what do you mean with "keeps crashing" - if the ide runs but suddenly crashs while editing - that sounds like some problems. If the ide just wont start - try to run it from the commandline. In your response please include what kind of "crash" you mean.

Last edited 2012

Grisu

(Posted 2012) [#8]

@Brucey:
The IDE keeps "hanging" after I use the scroll bar to scroll down in my code.
Maybe it got sick of my code... XD

P.S.: Nice to see you back on the forums!

@Derron: Ups, wrong link. Fixed. Please download again.

Yes, from this site. But I use the default MaxGUI module. Nothing from Brucey. Except a modified fmod module.
OS: Ubuntu 12.10 (32 Bit)

Last edited 2012

Derron

(Posted 2012) [#9]

Your testfile works (after I installed your .so file and "ln -s"ed it).

Although it looks a bit like a nagscreen overlaying the grayish background until the "radio stations" are loaded...afterwards the image is drawn properly).
All stays in background if i defocus the app (bring another to the front).

bye
Ron

edit: i am running linux mint 13 64bit here... so it is using gtk like yours. Did you check it in a virtual machine (booting a live cd)? So you can easily check numerous OS.

edit2: do you have compositor activated (shadows around your windows etc.) - test if running flawless if deactivated - maybe something with your opengl is interfering.

Last edited 2012

Brucey

(Posted 2012) [#10]

Nothing from Brucey

Yeah, cuz that would be the death knell of anything ;-)

Derron

(Posted 2012) [#11]

Lol.

Maybe he meant: brucey is using magic - and all know, magic only works if you believe in it :D.

edit: thanks for updating your ***-blog @ brucey

Last edited 2012

Grisu

(Posted 2012) [#12]

@Brucey:
More like: If it isn't working -> it's the result of my bad code and not yours.... ;)

@Derron:
Thanks for the feedback.

You need to play a station and then defocus the window!

Guess I need to figure out on how to fix the startup screen. It should look like this. At least it does under Windows and Mac:

Could you please check the updates function. Is it working for you?

From the station screen window -> go to "About" -> "Check Updates".

I'm using a virtual box with a fresh installation for testing purposes.

Last edited 2012

Derron

(Posted 2012) [#13]

updates function (icons are hmm ... may be enable added text labels by default - i just found the correct ones as no others were present :D).

i pressed update:
ahh ... ok ... something changed now (last run was ok).

I pressed update - surfed to your PRP-website and re-tabbed into the app, the main panel of the app is not redrawn (presenting a bit of my application switcher). Application exiting is also not possible (xkill it)
Third try: everything works.

I defocused the app while playing hitz.fm or so ... no problem.

VirtualBox ... hmm maybe that is a reason as their opengl-"driver" is not 100% capable of doing opengl :D

for you in German (to avoid misunderstandings):
Beim Alt-Tabben hat es den Fensterinhalt nicht neu gezeichnet (auch nicht nach Minimieren etc.). Irgendwo hat es wahrscheinlich auf die Antwort von einem Server gewartet und damit die Applikation eingefroren.
Abspielen ging wie erwartet, auch Kuenstlerinformation wurde korrekt aktualisiert.
Probiere es mal auf einem realen Linux-Desktop, VirtualBox ist nicht komplett kompatibel zu OpenGL ... bzw muss man die 3D-Beschleunigung aktivieren (abgesicherter modus)

bye
Ron

edit: please put the data fetching in a different thread... apps not responding while waiting for data... this is really oldschool. And as enduser you do not know whether the app is frozen or what happened now.

Last edited 2012

Derron

(Posted 2012) [#14]

update: one of your apps instances still run in the background hogging up 25% of one of my cores...

GUI-Apps should stay < 5%.

bye
Ron

Grisu

(Posted 2012) [#15]

The app instance still running should be the one that crashed. I doubt my app will ever use 25% of a PC core.

I uploaded another version (link at post #5). This version should fix your ALT-TAB and Minimize issues. In addition I updated some recently broken station files.

I don't know how to handle threads in bmx. Doesn't this require Brucey's "Alice in Wonderland" compiler? ;)

Derron

(Posted 2012) [#16]

It just needs "compile threaded" and the special thread-commands.

As your app is MVC or "user control vs audio output" it is made for threading.

Equalizer-animation etc would be done in the main thread, audio and data fetcher in other threads.

@25%: if an app crashes, it should not increase its hunger for cpu time... so something else happened there.

bye
Ron

Brucey

(Posted 2012) [#17]

Doesn't this require Brucey's "Alice in Wonderland" compiler? ;)

Hey! Who the � is Alice? :-p

As Derron says, anything that isn't GUI related could be done in a background thread. FMOD should already be doing that anyway, and you can continue to call update() in the main thread.

Any fetches for data you are doing yourself to the internet, should be done as a background task, so that the UI remains responsive at all times. Once the stuff is fetched, you can set a flag or something that you look at every so often and react to it - updating a list or some such.

Grisu

(Posted 2012) [#18]

So all I have to do is to is to shift the "file download" below to an extra thread and wait till it's done?

Function HTTPGetFile:Int(url$ , filePath$)
	Url$ = url$.Replace("http://", "http::")
	Local bank:TBank=LoadBank(url)
	If bank <> Null Then 
	?macos
        SaveBank bank,CurrentDir()+"/"+filePath$
	?
	?Win32
	   SaveBank bank,CurrentDir()+""+filePath$
	?
	?Linux
	   SaveBank bank,CurrentDir()+"/"+filePath$
	?
         Return 1
      Else 
         Return -1
      EndIf
End Function

Or is that approach too simplistic?

Last edited 2012

Derron

(Posted 2012) [#19]

Instead of using different "saveBank"-styles just decide globally what kind of directory limiters you use (slash or backslash).

The Approach is:

?Threaded
Import brl.Threads
?

Type TDataFetcher
	field result:string = "empty"
	field data:TList = CreateList()
	global instance:TDataFetcher


	?Threaded
	global MutexContentLock:TMutex = CreateMutex()
	global LoadLock:TMutex = CreateMutex()
	global LoadThread:TThread
	?


	Method New()
		self.instance = self
	End Method

	'singleton like
	Function GetInstance:TDataFetcher()
		if TDataFetcher.instance = null then TDataFetcher.instance = new TDataFetcher

		return TDataFetcher.instance
	End Function

	'threadable function that loads data
	Function LoadData:object(data:object)
		local url:string = string(data)

		local _self:TDataFetcher = TDataFetcher.GetInstance()
		'fetch data
		_self.result = _self.MyHTTPGetFile(url, "test.txt")
		?Threaded
			'if threaded - set a mutex so other threads
			'or main know whether it is safe to change a
			'value -> it is like an "isUsed"-variable
			'additionally: LockMutex waits until it can
			'lock -> it waits until no other is blocking
			LockMutex( _self.MutexContentLock )
		?

		_self.data.addLast( _self.result )
		print "added Data - result: "+_self.result

		?Threaded
			UnlockMutex( _self.MutexContentLock )
		?
	End Function

	'do it ... if compiled with threads
	'the "LoadData"-Call is done in a thread
	Method LoadRemote(url:string="")
		?Threaded
			if not self.LoadThread OR not ThreadRunning(self.LoadThread)
				print "create thread"
				self.LoadThread = CreateThread(TDataFetcher.LoadData, url)
			endif
		?not Threaded
			self.LoadData(object(url))
		?
	End Method

	Method MyHTTPGetFile:int(url:string, path:string="")
		url = url.Replace("http://", "http::")
		local in:TStream = ReadStream(url)

		While Not Eof(in)
			self.data.addLast( ReadLine(in) )
		Wend
		CloseStream in
	End Method


	Function HTTPGetFile:Int(url:string , filePath:string)
		print "HTTPGetFile"
		url = url.Replace("http://", "http::")

		Local bank:TBank = LoadBank( url )

		local delimiter:string = "/"
		?Win32
		   delimiter = ""
		?

		If bank <> Null
'			SaveBank( bank,CurrentDir()+delimiter+filePath )
			Return 1
		Else
			Return -1
		EndIf
	End Function

End Type


local test:TDataFetcher = new TDataFetcher
print "Startup"
print "Before LoadRemote..."
test.LoadRemote("http://www.blitzmax.com")
print "After LoadRemote..."
print "Waiting 5 seconds then exiting"
Delay(5000)
print "data lines: "+test.data.count()
print "Bye"

Maybe that helps

Grisu

(Posted 2012) [#20]

It took a while, but helped... :)
The file downloads now use a child thread.

Not sure if these changes solve your issue a bit though: http://www.mediafire.com/?aks4rel8y239pru

I had to use "WaitThread()" in order to determine when the child thread is finished. The visible "input lag" is a result of the "SaveBank()" command it seems.

I also recognised that the translation phrases are somewhat "broken". :/
Is there a way to force Linux to load a plain text file in the "Windows format". So German umlauts etc are displayed correctly?

Brucey

(Posted 2012) [#21]

I had to use "WaitThread()" in order to determine when the child thread is finished.

Well, that mostly defeats the purpose, doesn't it?

Generally you need a design change if you introduce multi-threading that is doing things like loading stuff - i.e. your program can continue doing its thing before the file has finished loading, and not just pausing.
If you must pause anyway, there's not much different in using a background thread.

What's a Windows format file? You mean like Windows1256 code pages or such things? Better to have your files in UTF-8 format, otherwise you need to know what the format of the file is, and convert it as it loads. I think the TextStream module can handle the basic UTF-8 ones.
If you need more support, something like BaH.libiconv can help (since that is what it is designed to do).

Derron

(Posted 2012) [#22]

umlauts (�� etc.) : just save your files using utf8. Like Brucey said: textstream can handle this. For umlauts within the sourcecode, that files have to get saved as utf8 too (simetimes it is needed to even store with BOM...).

Multitasking: again like Brucey said, do not wait for the results to be there.

Example: Store all stations in a list
If the Backgroundfetcher modifies the data, it does:

LockMutex( _self.MutexContentLock )
list.addLast(data)
UnlockMutex( _self.MutexContentLock )

That means: set a value to true - or wait until I am able to do so (lockmutex waits until sucessfull).

In your main thread - you do the same (although read access is easier)
Your main thread should also try

LockMutex( _self.MutexContentLock )
'work with the data list - eg read modify etc.
UnlockMutex( _self.MutexContentLock )

As long as the backgroundfetcher locks the "MutexContentLock", the main thread will wait with list processing until the fetcher has unlocked.

That lock will only be used when changing shared data (the list), the real fetching work is done without interrupting the main thread.

So you "SaveBank" outside of the "lock-action-unlock" chain - and no disturbing pause-effect will be shown to the user.

Your "broken phrases" - that is because I print to the output of the whole app. Text isnt printed "at once" - imagine each char is sent to the outputbuffer alone. The Main thread is sending 2 chars and baemm now the sub thread is sending 1 of its 10 chars... the result is a mixture :D.
If you buffer your output to a list and print out that list "time by time" (with lock mutex etc when adding) you circumvent this.
Call it "TBufferedOutput", add a field list, a mutex and "addLog"/"outputNew"-Method ... there you go.

Another hint for "testing for updates" ... add:

 field hasToUpdate:byte
 ?Threaded
 field hasToUpdateMutex:TMutex
 ?
 '....in your update methods
 LockMutex(hasToUpdateMutex)
 hasToUpdate = 1
 UnlockMutex(hasToUpdateMutex)

And check versus that boolean in your main loop.... I am not sure but I think read access to a byte could be safe even without a locking mutex ... but better add one :D.

Short Summary in German:
- WaitThread brauchst Du nicht, einfach mit "LockMutex" arbeiten, dass wartet solange, bis der Mutex freigegeben wurde (UnlockMutex), dass heisst waehrend dein Subthread in die Datenliste schreiben will, lockt er einen mutex, schreibt rein und gibt den Mutex wieder frei.
Dein Mainthread lockt vorm lesen der Liste ebenfalls den Mutex - sollte in dieeeessem Moment der Subthread auch schreiben - wartet "LockMutex" bis "UnlockMutex" aufgerufen worden ist. Dadurch kann es keine konkurrierenden Situationen geben - einfach "LockMutex" nutzen.

bye
Ron

PS: write and I will code a small "RSS"-Fetcher-App which fetches data in the background etc... hmm think I will do that even now

Last edited 2012

Derron

(Posted 2012) [#23]

Ok there was 1 small mistake in my prior code.
The DataFetcher locked a mutex just for adding something to the data although the loader-method already manipulated the list... don't do that :D

The following piece of code emits an event in case of remote data was loaded.
If eventemit is somehow not threadsafe - replace that part with the thing I described one post ago (hasToUpdate-Flag).

Framework MaxGUI.Drivers
'Import bah.gtkmaxgui
Import BRL.GLMax2D
Import BRL.EventQueue
Import BRL.retro
Import BRL.timer
?Threaded
Import brl.Threads
?


Global App:TApp	= New TApp
'run the app - and loop etc.
App.Run



Type TApp
	Global instance:TApp		= Null			'singleton principle
	'gfx/gui
	Global guiWindow:TGadget	= Null
	Global guiMainPanel:TGadget	= null
	Global guiFileMenu:TGadget	= Null
	Global guiTextarea:TGadget	= Null
	Global guiButton:TGadget	= Null
	Global EVENT_FASTKEYHIT:Int	= AllocUserEventId("fastkeyhit") 'faster key response
	Global EVENT_REMOTEDATA:Int	= AllocUserEventId("remotedata")

	field data:TDataFetcher = new TDataFetcher
	field dataFetchInterval:TTimer = CreateTimer(1.0/60.0) 'fetch every 60 seconds

	Const MENU_OPEN:Int		= 101
	Const MENU_EXIT:Int		= 102

	Method New()
		Self.instance = Self
	End Method

	Function GetInstance:TApp()
		If TApp.instance Then Return TApp.instance
		Return New TApp
	End Function

	Method Run:TApp()
		guiWindow = CreateWindow( "Test", 100, 100, 800, 550, Null, WINDOW_MENU|WINDOW_TITLEBAR|WINDOW_RESIZABLE|WINDOW_STATUS|WINDOW_CLIENTCOORDS )

		'panel for main+right, pin at all corners
		guiMainPanel =  CreatePanel(0,0,ClientWidth(guiWindow),ClientHeight(guiWindow),guiWindow,PANEL_GROUP, "TestPanel")
		guiMainPanel.SetLayout( EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED )

		guiTextarea = CreateTextArea( 0, 30	, ClientWidth(guiMainPanel), ClientHeight(guiMainPanel), guiMainPanel,  TEXTAREA_WORDWRAP )
		guiTextarea.SetLayout( EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED,EDGE_ALIGNED )
		guiTextarea.AddText("Hi :D")

		guiButton = CreateButton( "Reload", 2, 2, 180, 28, guiMainPanel )

		'FileMenu
		guiFileMenu = CreateMenu("&File",0,WindowMenu(guiWindow))
		CreateMenu( "&Open Something"	, MENU_OPEN	,guiFileMenu	, KEY_V, MODIFIER_COMMAND )
		CreateMenu( ""						, 0			,guiFileMenu )
		CreateMenu( "&Exit"				, MENU_EXIT	,guiFileMenu	, KEY_F4, MODIFIER_COMMAND )
		'add menus
		UpdateWindowMenu(guiWindow)

		While True
'			ActivateGadget(guiCanvas)
			WaitEvent
			Select EventID()
					'testcase - timed canvas update (for background-draw-test)
					Case EVENT_TIMERTICK
						if EventSource() = TApp.GetInstance().dataFetchInterval
							guiTextarea.AddText("fetching data "+Millisecs())
							print "fetching data" + Millisecs()
							TApp.GetInstance().data.LoadRemote("http://www.blitzmax.com")
						endif
					Case EVENT_REMOTEDATA
							'event data is our data list
							local data:TList = TList(EventExtra())
							guiTextarea.SetText("") 'clear
							'for local line:string = eachin TApp.GetInstance().data.getData()
							'	guiTextarea.AddText(line) 'clear
							'Next
							'for local line:string = eachin data
							'	guiTextarea.AddText(data.count() ) 'clear
							'Next
							guiTextarea.AddText("got new data" + Millisecs()+"~n~n")
							guiTextarea.AddText(string(data.First()) )


					Case EVENT_MENUACTION
						Select EventData()
							Case MENU_EXIT
									End
							Case MENU_OPEN
									'open file
									'in case of changes:
									'guiScrollableCanvas.Redraw()
						End Select
					'checking both events -> faster response on "keep pressed"
					Case EVENT_KEYREPEAT
						Self.HandleNavigatorKeys(CurrentEvent)
					Case EVENT_KEYDOWN
						Self.HandleNavigatorKeys(CurrentEvent)
					Case EVENT_GADGETACTION
						'check EventSource() against other interactable objects (eg lists)
						if EventSource() = self.guiButton
							'reset timer
							TApp.GetInstance().dataFetchInterval = CreateTimer(1.0/60.0)
							'manually run fetcher
							TApp.GetInstance().data.LoadRemote("http://www.blitzmax.com")
						endif
					Case EVENT_WINDOWCLOSE
						End
			End Select
		Wend

		Return Self
	End Method


	Method HandleNavigatorKeys(event:TEvent)
		If event.data = KEY_DOWN Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_DOWN))
		If event.data = KEY_UP Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_UP))
		If event.data = KEY_LEFT Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_LEFT))
		If event.data = KEY_RIGHT Then EmitEvent(TEvent.Create(EVENT_FASTKEYHIT,Null,KEY_RIGHT))
	End Method
End Type



Type TDataFetcher
	field _data:TList = null			'_data is private, do not read in other threads
	field data:TList = CreateList()		'public "data" field
	global instance:TDataFetcher


	?Threaded
	global MutexContentLock:TMutex = CreateMutex()
	global LoadLock:TMutex = CreateMutex()
	global LoadThread:TThread
	?


	Method New()
		self.instance = self
	End Method

	'singleton like
	Function GetInstance:TDataFetcher()
		if TDataFetcher.instance = null then TDataFetcher.instance = new TDataFetcher

		return TDataFetcher.instance
	End Function

	'threadable function that loads data
	Function LoadData:object(data:object)
		local url:string = string(data)

		local _self:TDataFetcher = TDataFetcher.GetInstance()
		'fetch _data
		_self.MyHTTPGetFile(url, "test.txt")

		?Threaded
			'if threaded - set a mutex so other threads
			'or main know whether it is safe to change a
			'value -> it is like an "isUsed"-variable
			'additionally: LockMutex waits until it can
			'lock -> it waits until no other is blocking
			LockMutex( _self.MutexContentLock )
		?

		'store private data in public data - others may read that
		_self.data = _self._data

		?Threaded
			UnlockMutex( _self.MutexContentLock )
		?
		'emit event so others know we finished loading
		EmitEvent( TEvent.Create(TApp.EVENT_REMOTEDATA, _self, 0,0,0,0,_self.data) )
	End Function

	'this uses another temporary variable but keeps the LockMutex-things
	'out of the App-Mainloop
	Method GetData:TList()
		local result:TList = null
		?Threaded
			LockMutex( self.MutexContentLock )
		?
		result = self.data

		?Threaded
			UnlockMutex( self.MutexContentLock )
		?
		return result
	End Method

	'do it ... if compiled with threads
	'the "LoadData"-Call is done in a thread
	Method LoadRemote(url:string="")
		?Threaded
			if not self.LoadThread OR not ThreadRunning(self.LoadThread)
				print "create thread"
				self.LoadThread = CreateThread(TDataFetcher.LoadData, url)
			endif
		?not Threaded
			self.LoadData(object(url))
		?
	End Method

	Method MyHTTPGetFile:int(url:string, path:string="")
		self._data = CreateList() 'new one

		url = url.Replace("http://", "http::")
		local in:TStream = ReadStream(url)

		While Not Eof(in)
			'store in our private data
			self._data.addLast( ReadLine(in) )
		Wend
		CloseStream in
	End Method

End Type

Brucey

(Posted 2012) [#24]

Derron, your German is almost as good as your English ;-)

EVENT_REMOTEDATA .. cool :-)

Derron

(Posted 2012) [#25]

Just thought about the event-params.

Imagine only "references" are sent with TEvent ... if then modifaction takes place problems may arise.
Better/Safer it seems to make real copies.
Instead of making real copies on emit one should consider doing that on "receive". Why? Imagine the datafetcher-listener being kind of an module. Why copying data (->work) each time the event happens. Just do that work in case one listens to that event.

In our case the "simplest" thing is: lockmutex on remotedata-event, copy the list entry by entry, unlock an mutex.

@German: may be the reason is my place of birth (still living here in Chemnitz, Saxony, Germany).
But thanks for the compliment concerning English.

bye
Ron

Grisu

(Posted 2012) [#26]

Thanks again for that input.

@Threads:
Is the maximum number of threads limited automatically? E.g. the available cpu cores on the current system?

I found some Windows code to detect the maximum number of CPUs available, but nothing cross-platform.

If I start to download 1000 small files (each one thread), the file server might not like this at all. Therefore I would need to limit the current threads to 8 or so.

@Localisation files:
I'm about to dumb all my localisation code and start fresh.

I found Bucey's old "Locale" module. The examples run fine under Windows, but still no luck under Linux (see screenshot below).

Am I missing some core Ubuntu library that causes the problem? What else can it be.

Brucey

(Posted 2012) [#27]

Try running the example on the console/terminal, and see what the results look like.

On Linux, DO NOT trust the Output window in the IDE. Really.

Ole JR

(Posted 2012) [#28]

It's the MaxIDE that fail to output correct locals..
Run the example in a console, and it will be fine.

Derron

(Posted 2012) [#29]

Or use my Geany-version for correct locales output.

@threads.
NEVER run 1000 threads vs remotely located URIs... Germany does not like "criminal acting" and 1000 concurrent requests are utilizing more power than needed -> you can get problems with "ddos"ing - although you do not do that ofc.

But hey...why 1000 threads to load the data...
Just let one datafetcher-instance do the job, url by url.
Each time it finished loading the remote data an event is emitted - and your gui can fetch the "new" data (meanwhile the datafetcher will fetch the next one ...).
You can split that fetching into some more worker threads, but this only improves the situation if remote servers are slow and take ages to respond.
-> use <5 workers, that will do.
-> you can kill the workers if they do not bring results (call it "timeout" :D)

so you do and need:
- set of workers (data fetchers)
- set of "toLoad"-Elements
- each time a worker finishes fetching data: emit event with data
- each time you get an "remotedata"-event, you propagate that fetcher to load another element (mark it as used or remove it from the list)

so you end up with a kind of "loop" which finishes alone if no data is left in the "toLoad"-List.
To Start it - manually kick off remote-Loading for each worker
(for local worker:TWorker = eachin workerList ...)

You can even make it more self-contained with the workers listening to "newToLoadElement"-Events ... but that is overkill.

bye
Ron

edit:
PS: this is nearly the same method I load my Ressources (Ressourcemanager can be threaded or not... in case of threaded it loads images as tpixmap and appends them to another list which is loaded from the main thread - as those are allowed to access the gfx-context)

Last edited 2012

Grisu

(Posted 2012) [#30]

I need to see the phrases inside a GUI app.

As this quick & dirty example shows, it's not working.

' createtextarea.bmx

Import MaxGUI.MaxGUI
Import BRL.StandardIO
Import MaxGui.Drivers
Import BaH.Locale

Strict

'Load the BLF
LoadLocaleFile("example_01.blf")

' get a string array of loaded language codes
Local locales:String[] = GetAvailableLocales()

Global window:TGadget = CreateWindow( "My Window", 130, 20, 400, 400 )

Global textarea:TGadget = CreateTextArea( 0, 0, ClientWidth(window), ClientHeight(window), window )
	SetGadgetLayout( textarea, EDGE_ALIGNED, EDGE_ALIGNED, EDGE_ALIGNED, EDGE_ALIGNED )
	ActivateGadget( textarea )

' Select the entire third (index: 2 [base-0]) line.
SelectTextAreaText( textarea, 2, 1, TEXTAREA_LINES )

' Output the properties of the current text selection (should be 1, 1 as set above).
Print "TextAreaCursor(): " + TextAreaCursor( textarea, TEXTAREA_LINES )
Print "TextAreaSelLen(): " + TextAreaSelLen( textarea, TEXTAREA_LINES )

Print "Languages = "
For Local i:Int = 0 Until locales.length
	AddTextAreaText(textarea, GetLanguage(locales[i])+Chr(10))
	Print "            " + locales[i] + " - " + GetLanguage(locales[i])
Next

Print "~n"

' for each language code, output all the numbers
For Local i:Int = 0 Until locales.length
	SetCurrentLocale(locales[i])
	AddTextAreaText(textarea, Chr(10))
	AddTextAreaText(textarea , GetLanguage(locales[i]) + Chr(10) )
	AddTextAreaText(textarea,"~t"+GetLocaleText("one")+Chr(10))
	AddTextAreaText(textarea,"~t"+GetLocaleText("two")+Chr(10))
	AddTextAreaText(textarea,"~t"+GetLocaleText("three")+Chr(10))
Next

SetCurrentLocale(locales[5]) 'russian language
SetStatusText (Window, GetLocaleText("one"))

While WaitEvent()
	Print CurrentEvent.ToString()
	Select EventID()
		Case EVENT_WINDOWCLOSE
			End
		Case EVENT_APPTERMINATE
			End
	End Select
Wend

Derron

(Posted 2012) [#31]

I do not have bah.locales installed (and are on my way to dads birthday... and my rum with fruits from the summer is awaiting to get unboxed :D).

But: is the ".blf"-file ascii or binary? if it is the first: save it with utf8-encoding.
Save your sources with utf8 if you have strings in there too.

To ensure the ladder one:
textareaObj.AddText("Ehne Mehne M�he, wir alle lieben K�he")

If that is print correctly (remember to save your .bmx with utf8 - that isn't done with the normal IDE) than you know what to do.
Geany, Gedit, etc - they are able to safe as utf8-encoded textfile.

bye
Ron

Grisu

(Posted 2013) [#32]

Here we go again.
I've managed to implement the new updater code. While under Windows it runs fine, no more stuttering while the download is in progress. The bad news: under Linux it doesn't seem to catch the custom event. So the updater "hangs" while downloading the first file from the server. :/
I added a DetachThread to your code in order to prevent a memory leak.

Newest Build:
http://www.mediafire.com/?z3b3cibb23h5hgo

Sourcecode:

Type TDataFetcher
'	Field _data:TList = Null			'_data is private, do not read in other threads
'	Field data:TList = CreateList()		'public "data" field
	Global instance:TDataFetcher


	?Threaded
	Global MutexContentLock:TMutex = CreateMutex()
	Global LoadLock:TMutex = CreateMutex()
	Global LoadThread:TThread
	?

	Method New()
		Self.instance = Self
	End Method

	'singleton like
	Function GetInstance:TDataFetcher()
		If TDataFetcher.instance = Null Then TDataFetcher.instance = New TDataFetcher
		Return TDataFetcher.instance
	End Function

	'threadable function that loads data
	Function LoadData:Object(data:Object)
		Local url:String = String(data)

		Local _self:TDataFetcher = TDataFetcher.GetInstance()
		'fetch data
		_self.HTTPGetFileStation(url)

		?Threaded
			'if threaded - set a mutex so other threads
			'or main know whether it is safe to change a
			'value -> it is like an "isUsed"-variable
			'additionally: LockMutex waits until it can
			'lock -> it waits until no other is blocking
			LockMutex( _self.MutexContentLock )
		?

		'store private data in public data - others may read that
		'_self.data = _self._data

		?Threaded
			UnlockMutex( _self.MutexContentLock )
		?
		'emit event so others know we finished loading
		EmitEvent( TEvent.Create(EVENT_REMOTEDATA , _self , 0 , 0 , 0 , 0))', _self.data) )
		DetachThread(LoadThread)  
	End Function
Rem 
	'this uses another temporary variable but keeps the LockMutex-things
	'out of the App-Mainloop
	Method GetData:TList()
		Local result:TList = Null
		?Threaded
			LockMutex( Self.MutexContentLock )
		?
		result = Self.data

		?Threaded
			UnlockMutex( Self.MutexContentLock )
		?
		Return result
	End Method
end rem
	'do it ... if compiled with threads
	'the "LoadData"-Call is done in a thread
	Method LoadRemote(url:String="")
		?Threaded
			If Not Self.LoadThread Or Not ThreadRunning(Self.LoadThread)
				'Print "create thread"
				Self.LoadThread = CreateThread(TDataFetcher.LoadData, url)
			EndIf
		?Not Threaded
			Self.LoadData(Object(url))
		?
	End Method


	Function HTTPGetFileStation:Int(url:String)
		'Print "HTTPGetFile "+STATIONS_DIR+url
  		Local Dest$ = STATIONS_DIR+url
		url = ini_update_server+"stations/"+url
            url = url.Replace("http://", "http::")
          
        'Print "Loading "+url
		Local bank:TBank = LoadBank( url )

		If bank <> Null
		    'Print "Saving "+CurrentDir()+"/"+Dest
			SaveBank( bank,CurrentDir()+"/"+Dest )
			Return 1
		Else
		    'Print "Error: Saving "+CurrentDir()+"/"+Dest
			Return -1
			 
		EndIf
	End Function

End Type

As the BMX IDE can't handle utf8 properly (= i.e. switching from Win7<->Linux<->Win7) I'm going to use external files for all my text labels. Not very fancy but bulletproof.

Does Linux require some special code in order to display an image icon (16x16) for an app? I'm using the code from the bmx ide source, but the icon isn't displayed?!

?linux
SetGadgetPixmap (MainWindow , LoadPixmap("Incbin::./data/prp16.png"), GADGETPIXMAP_ICON) 
?

Last edited 2013

Derron

(Posted 2013) [#33]

- DetachThread - we reuse the same variable for the threads, so this should not be that big point

- The sample code does not show how you implement your custom event. As my "EVENT_REMOTEDATA" was catched like it should - I think you did something differently than me

- Icon: I think displayed icons are stored externally eg usr/share/icons and the system tools will have a look there (in your case nautilus as file explorer when displaying the desktop or the file explorer itself)

edit:
- "the updater hangs" - the good thing is: only the updater hangs, the rest of the gui and app will keep being responsive :p.

bye
Ron

Last edited 2013

Grisu

(Posted 2013) [#34]

Here's my main loop. Can't figure out why it stops.
Sometimes 3 files are downloaded. Sometimes 6 or more. - Then it "hangs" = no event incoming.

1. I create a custom array (FileArray) which holds all filenames that need to be downloaded.
2. Then I start one download thread in order to get things started.
3. Enter the main loop
4. Catch the EVENT and UPDATE all treeview informations / progress labels
5. Start the next thread until all files were downloaded.

Local FileArray:TStationFile[maxcounter] ' Static Index of all files that need to be downloaded 
Local Index:Int = 0

' Create Index entries from the Downloadlist
For tmp = EachIn download_list
    FileArray[Index]=New TSTATIONFILE
    FileArray[Index].name = Get_StationName(tmp) 
    FileArray[Index].size_str = Get_Filesize(tmp)
    FileArray[Index].size = Int(FileArray[Index].size_str)
    If Len(FileArray[Index].size_str) > 3 Then FileArray[Index].size_str = Left(FileArray[Index].size_str , Len(FileArray[Index].size_str) - 3) + "." + Right(FileArray[Index].size_str , 3) 'adding "."
    'Print "added ["+Index+"] : "+FileArray[Index].name	 
    If FileType( STATIONS_DIR + FileArray[Index].name ) = 1 Then FileArray[Index].ft=1 Else FileArray[Index].ft=0 
    Index=Index+1
Next

 
' Check and dowload new stations
'     SetGadgetText (label_upd1,"")
     SetGadgetText (label_upd3,"0%")
     SetGadgetText (label_upd2,"")
     'SetGadgetText (label_upd2,GUI_LABELS_UPDATER[5]+"...")
     SetGadgetText (label_upd4,"")
     SetGadgetText (label_upd5,"")

     Index=0
     'Print "Init Startup Child Thread 1. File ----- "+FileArray[Index].name    
     TDataFetcher.GetInstance().LoadRemote(FileArray[index].name)
Repeat
      Select EventID()
        Case EVENT_GADGETLOSTFOCUS
            RedrawGadget (UpdatePanel)

        Case EVENT_WINDOWCLOSE
           RedrawGadget(MainWindow)
           	End_PRP()

	 Case EVENT_REMOTEDATA
     		Print "EVENT_REMOTEDATA: " + FileArray[Index].name + " finished loading!"
           Print ""
            If FileArray[Index].ft=1 Then
               AddTreeViewNode FileArray[Index].name , tree_upd , - 1 , FileArray[Index].size_str + " Bytes"
               ModifyTreeViewNode (tree_upd, GUI_LABELS_UPDATER[2]+" ("+CountTreeViewNodes(tree_upd)+")") 
            Else
                AddTreeViewNode FileArray[Index].name, tree_new, -1,FileArray[Index].size_str+" Bytes"
   		     ModifyTreeViewNode (tree_new, GUI_LABELS_UPDATER[1]+" ("+CountTreeViewNodes(tree_new)+")") 
            EndIf 
	      stations_todo_size:Int=stations_todo_size-FileArray[Index].size
       	todo_size:String=(stations_todo_size/1024) ' KB transformation 
	      If Len(todo_size) > 3 Then todo_size=Left(todo_size,Len(todo_size)-3)+"."+Right(todo_size,3) 

      	counter:Int = counter + 1
       	Progress#=Float(counter)/Float(maxcounter)
       	Progress_curr:Int=Int(Progress*100)

	  	SetGadgetText (label_upd3,progress_curr+"%")
	  	UpdateProgBar (upd_progbar,progress)

        	SetGadgetText (label_upd4 , todo_size)
        	SetGadgetText (label_upd5 , "kb")

    		Index=Index+1

           If counter < Maxcounter Then
             Print "Next file ----- No. "+Index+": "+FileArray[Index].name+" / Counter: "+counter+" INDEX: "+Index
             TDataFetcher.GetInstance().LoadRemote(FileArray[index].name)
           EndIf

 	  Case BUTTON_ACTION
            'A BUTTON TO STOP UPDATE IF IT TAKES TOO LONG      
           Select EventSource()
           Case Button_Update_back
                   If ini_first_autoupdate=0 Then ' dont show the about panel if updater is running at startup of prp 
                    ShowGadget(AboutPanel)
                    HideGadget(UpdatePanel)
                   Else 
                   EndIf                  

                   Exit 
           End Select

    End Select

    WaitEvent
Until counter=maxcounter ' all files are downloaded or user hits EXIT button

Font issue:
Do you know of a way to force BMX to use an arial like font under Linux?
In former MaxGUI releases bold text labels were shown correctly.
The default font seems to be different and the labels are not bold anymore.

New vs. Old:

Last edited 2013

Derron

(Posted 2013) [#35]

fonts: this may be caused by fltk as it is not using the gnome-settings all in all - but prior compilates (using the gtk bindings) might have done that.
Maybe you have done a completely new installation so some "user configs" are missing.

' Create Index entries from the Downloadlist
For tmp = EachIn download_list

Hope you compile (super)strict and declared tmp before...
codelines like

Progress_curr:Int=Int(Progress*100)

Do not make me believe in that (why declaring an variable as :int if it is already declared as an int before (<-- assumption)

So - the loop looks like:
1. first call to load a file
2. loop
2.1 loop event "remotedata" received - process it and start new thread

Question: is Print "EVENT_REMOTEDATA: "... working but Print "Next file ----- No... missing ?

Again I want to cite myself: I am not sure whether the event-system is thread safe or not.
Instead of using the EmitEvent-Style, you could also have a global within
TDataFetcher -> Field newRemoteData (or hasToUpdate ...).
You then just check versus that integer-field whether to do something or not.
You may use Events there... but in case of events getting "lost" you also are able to use a timer and check each timer_tick if the "hasToUpdate" variable is 1 or 0 or whatever.

You could also have integer fields holding information about "used" DataFetchers and a "available" amount of DataFetchers.
Check whether the list of still-to-fetch-data is longer than the amount of unused DataFetchers - and then give them something to do.

Just think a bit about it, then you will surely get around it by yourself.

bye
Ron

edit:
a) keep in mind that "singleton" (getinstance() ...) means: there is only 1 instance of that class available.

b) if events are not threadsafe do not try to do:
...loadRemote(a)
...loadRemote(b)
They all would Emit the RemoteData-Event.

Now imagine 2 of that loadRemote-threads finish nearly the same and try to emit the event... in both cases the event-system adds eg. event number 3, will then increase the next "ID" to 4 - but before doing that the next thread adds his event as 3rd and wants to increase to 4. So 2 events become 1.

Making the event-system taking care of threads is really "easy" - like we have done it with the datafetcher, just lock a mutex before changing eg. a list - or looping through it (do not know much about the iterator system).

A custom event-system is no magic and also allows firing an event in just that moment instead of waiting for an "WaitEvent" or "UpdateEvents" call.
This is nifty in case of immediate calls instead of "1 update loop later".

Last edited 2013

Grisu

(Posted 2013) [#36]

I've created a simple source code file that runs fine under Windows.

There has to be a simple issue I'm unable to detect that stops it from working under Linux. - If anyone wants to give it a shot under Linux or Mac, be my guest.

Source code (70 kb):
http://www.mediafire.com/?idbgxbvcbemc77i

Derron

(Posted 2013) [#37]

Import BRL.GLGraphics
Import BRL.GLMax2D
--> imported twice

Import BRL.retro
--> is missing but added because you use gman.zipengine (i commented that out)

you commented out "?Threaded" but left the corresponding "?" uncommented alive.

you use conditionals for win32 and rest but are using:
Print "Saving "+CurrentDir()+""+Dest
without proper directory-separator-variable.

But to give some real help:

You may recognize that my downloadmethod did not use LoadBank but yours does... there you got your problem.

bye
Ron

edit:

[bbcode]
Function HTTPGetFileStation:Int(url:String)
'Print "HTTPGetFile "+STATIONS_DIR+url

local delimiter:string = "/"
?Win32
delimiter = ""
?

Local filePath:string = CurrentDir() + delimiter + STATIONS_DIR + url

url = (ini_update_server+"stations/"+url).Replace("http://", "http::")

Print "Loading stream "+url
local inStream:TStream = ReadStream( url )
local outStream:TStream = WriteStream( filePath )

while not Eof(inStream)
outStream.writeByte( inStream.readByte() )
Wend
Print "processing stream done"
inStream.close()
outStream.close()

if filesize( filePath ) = -1
Print "Error: Saving File to " + filePath
Return -1
else
Print "Saved File to "+ filePath
Return 1
EndIf
End Function
[/bbcode]

should work - although there is a better method.

edit2:
I now understood why the event firing was working but "slow" (you had to wait a second until the next thread starts etc.)
It is because no event happens the event system will care of.

So before your "repeat"-loop (after the first eventemit) add:
[bbcode]
'check every 10 ms - so make a timer tick to satisfy "WaitEvent"
local FasterEventCheck:TTimer = CreateTimer(1.0/100.0) 'check every 10ms
[/bbcode]
After the loop you will just have to delete the timer (so it wont tick needless :D)

This will make the event "timer tick" firing every 10ms - and therefor other events will get checked too.
So the problem of "waitEvent" is, that it relies on getting system events - and no custom ones.

There is no better reason to use a custom event manager with direct fire/trigger-ability in combination with a timer which triggers every 10ms (yeahh i know... power consumption ...).

Last edited 2013

jsp	(Posted 2013) [#38]

I am not sure whether the event-system is thread safe or not.

The event system is not thread safe.

Brucey

(Posted 2013) [#39]

I'm pretty sure something like libcurl can do background fetches of web data so you don't have to arse around in BlitzMax code to handle multiple threads of fetchyness.

Grisu

(Posted 2013) [#40]

Thanks for the help everyone! It's really appreciated.

@Derron: Under Windows you extra timer speeds up things. But on Linux the code "hangs" as I'm used to see it (with or without the extra timer). Don't want to imagine what the code does under a Mac OS.

@Brucey: I got libcurl from SVN. Could you please give a simple example on how to download and save a file from an url with it. I've checked the ones included without success.

Ole JR

(Posted 2013) [#41]

Very crude example of how to use libcurl.

SuperStrict

Framework BaH.libcurl
Import BRL.StandardIO
Import brl.bank
Import brl.bankstream

'Set to true to use bankstreams instead of strings
Const USE_BANKS:Int = False

Local curl:TCurlEasy = TCurlEasy.Create()

Local bank:TBank = CreateBank()
Local bankstream:TBankStream = CreateBankStream(bank)

curl.setOptInt(CURLOPT_VERBOSE, 0)

If USE_BANKS
	'Get the data into a TBank
	curl.setWriteStream(bankstream)
Else
	'Get the data into a String
	curl.setWriteString()
End If

'A site I'm currently watching for updates ;-)
'Change to whereever you want to get your data from.
curl.setOptString(CURLOPT_URL, "http://qtmax.googlecode.com/svn/trunk/qt.mod/build_notes.txt")

Local res:Int = curl.perform()
If (res<>0)
	Print("ERROR: Getting file!")
	End
End If

If USE_BANKS
	'Do whatever you do to get the data from the bank/bankstream
	Local test:Byte = PeekByte(bank, 0)
	DebugLog Chr(test)
Else
	' curl.toString() return the string from curl
	Local test:String = curl.toString()
	DebugLog test
End If

curl.cleanup()

End

Brucey

(Posted 2013) [#42]

The problem with TCurlEasy is it is blocking by default.

If you want to something more useful, you need to use the "TCurlMulti" interface, which is a non-blocking interface.

@Grisu : I rewrote your TestLoader2 example to use libcurl multi, and it appears to run fine on Mac (where I wrote it) and Linux (where I tested it). UI remains reasonably responsive while performing 8 concurrent downloads of prp files.
It runs off a 10ms timer tick which causes it to process the current downloads. As each download is complete, it closes that file stream, and pulls another one off the "todo" list, reinitialising the connection and commencing download.
When it is finished, it should clean itself up, but I haven't tested it 100%.

The only messy bit (IMO) is the tree update inside the process(), but since you want a live update of available files in your tree, I couldn't think of anywhere else to put it.

Well, you can see how well it performs for yourself when you run it.

On Linux you'll need libidn (dev) if you don't already have it to compile libcurl.

SuperStrict

Framework Maxgui.Drivers
?Threaded
	Import BRL.Threads
?
	Import BRL.Max2D
  	Import BRL.Filesystem
	Import Maxgui.ProxyGadgets
	Import Brl.EventQueue
	Import Brl.Timer     
	Import Gman.Zipengine ' <a href="http://www.gprogs.com/viewtopic.php?id=28" target="_blank">http://www.gprogs.com/viewtopic.php?id=28</a> ' 
	Import BRL.GLGraphics
	Import BRL.GLMax2D
	Import bah.libcurl
?Win32
	Import SRS.D3D11Max2D
	Import BRL.D3D9Max2D
	Import BRL.GLGraphics
	Import BRL.GLMax2D
?
Const MAINWINDOW_W:Int = 278 
Const MAINWINDOW_H:Int = 154
Const BUTTONSIZE:Byte = 30
Const LABEL_STARTY:Byte = 3

Global EVENT_REMOTEDATA:Int	 = AllocUserEventId("remotedata")

Global ini_update_server:String  ="http://dl.dropbox.com/u/98035398/prp/" ' <- URL TO UPDATE SERVER
Const  PRP_UPD_LIST:String="prp_stationlist"
Global zrObject:ZipReader = New ZipReader
Global fileBuffer:TRamStream = Null

Const BUTTON_ACTION:Int = EVENT_GADGETACTION

?Win32
	Const STATIONS_DIR:String = "stations/"
?
?Not Win32 '= MAC + LINUX /
	Const STATIONS_DIR:String = "stations/"
?
?win32
	Const PNL_HEIGHT:Int = 22
	Const SLIDER_WIDTH:Int = 22
	Const WIN_MENU_FLAG:Int = 0
?
?macos
	Const PNL_HEIGHT:Int = 22
	Const SLIDER_WIDTH:Int = 22
	Const WIN_MENU_FLAG:Int = WINDOW_MENU
?	
?linux
	Const PNL_HEIGHT:Int = 22
	Const SLIDER_WIDTH:Int = 22
	Const WIN_MENU_FLAG:Int = 0
?

Local Style:Int = WINDOW_TITLEBAR | WINDOW_CLIENTCOORDS | WINDOW_CENTER | WIN_MENU_FLAG |WINDOW_ACCEPTFILES | WINDOW_RESIZABLE
Global MainWindow:TGadget = CreateWindow("Downlad Example", 0 , 0 , MAINWINDOW_W , MAINWINDOW_H , Null , Style)
	
Global UpdatePanel:TGadget = CreatePanel(0, 0, MAINWINDOW_W, MAINWINDOW_H, MainWindow)', PANEL_ACTIVE) 
SetGadgetLayout UpdatePanel, EDGE_ALIGNED, EDGE_ALIGNED, EDGE_ALIGNED, EDGE_ALIGNED
SetGadgetSensitivity(UpdatePanel , SENSITIZE_MOUSE)

Global label_upd1:TGadget=CreateLabel:TGadget("",2,LABEL_STARTY,MAINWINDOW_W-6-30,16,UpdatePanel,Label_LEFT) 
Global label_upd3:TGadget=CreateLabel:TGadget("",2+(MAINWINDOW_W-6-30),LABEL_STARTY,30,16,UpdatePanel,Label_RIGHT) 

Global upd_progbar:TGadget=CreateProgBar(2,LABEL_STARTY+16,ClientWidth(MainWindow)-4,16,UpdatePanel)
Global upd_Treeview:TGadget  = CreateTreeView(2,6+16*2,MAINWINDOW_W-3, MAINWINDOW_H-2-BUTTONSIZE-2-6-16*2, UpdatePanel) 

Global label_upd2:TGadget=CreateLabel:TGadget("",2,MAINWINDOW_H-3-PNL_HEIGHT,MAINWINDOW_W-3-BUTTONSIZE-4-60,PNL_HEIGHT,UpdatePanel,LABEL_LEFT) 
Global label_upd4:TGadget=CreateLabel:TGadget("",2+MAINWINDOW_W-3-BUTTONSIZE-4-130,MAINWINDOW_H-3-PNL_HEIGHT,120-18,PNL_HEIGHT,UpdatePanel,Label_RIGHT) 
Global label_upd5:TGadget = CreateLabel:TGadget("" , 2 + MAINWINDOW_W - 3 - BUTTONSIZE - 4 - 130 + 120 - 16 , MAINWINDOW_H - 3 - PNL_HEIGHT , 16 , PNL_HEIGHT , UpdatePanel , LABEL_LEFT)

Global Button_Update_Back:Tgadget = CreateButton("EXIT", MAINWINDOW_W-2-BUTTONSIZE,MAINWINDOW_H-1-BUTTONSIZE,BUTTONSIZE,BUTTONSIZE,UpdatePanel,BUTTON_PUSH)

SetGadgetLayout label_upd1, Null, Null, EDGE_ALIGNED, Null 'filename
SetGadgetLayout label_upd3, Null, Null, EDGE_ALIGNED, Null '% progress
SetGadgetLayout upd_progbar, Null, Null, EDGE_ALIGNED, Null ' ----- Bar
SetGadgetLayout upd_Treeview, Null, Null, EDGE_ALIGNED, EDGE_ALIGNED ' List
SetGadgetLayout label_upd2, Null, Null, Null,EDGE_ALIGNED 'status bar
SetGadgetLayout label_upd4, Null, Null, Null,EDGE_ALIGNED 'Size left
SetGadgetLayout label_upd5 , Null , Null , Null , EDGE_ALIGNED '+ "KB"

PRP_Check_StationUpdates()
End


' / Getfile functions for the new station updater ---------------------------------------------------
Function HTTPGetFile:Int(url$, filePath$)
	Url$ = url$.Replace("http://", "http::")
	Local bank:TBank=LoadBank(url)
	If bank <> Null Then 
        SaveBank bank,CurrentDir()+"/"+filePath$
        Return 1
      Else 
         Return -1
      EndIf
End Function

Function Get_StationName:String(tmp_str:String)
Local slashPos: Int = tmp_str.Find("=")
Return tmp_str$[..(slashPos)]
End Function

Function Get_Filesize:String(tmp_str:String)
Local slashPos: Int = tmp_str.Find("=")
Return tmp_str$[(slashPos+1)..]
End Function


' Type for the static Array
Type TStationFile
    Field name:String 
    Field size_str:String 
    Field size:Int
    Field ft:Int 'update or new file?
End Type

Function PRP_Check_StationUpdates()

	' Disable Treeview list during the update process to speed it up a bit and to avoid user confusion
	'DisableGadget upd_Treeview
	
	Local counter:Int 
	Local Progress:Float
	Local Progress_curr:Int=0
	Local file:String
	Local rel_counter:Int=0
	Local maxcounter:Int=0
	Local upd_list:String[]'=CreateList()
	Local download_list:TList=CreateList()
	Local curr_rel_number:Int
	Local curr_build:String
	
	SetGadgetText (label_upd1,"Downloading Station Info... Please Wait... ") ' Set label to DL Stationlist
	
	Local root:TGadget=TreeViewRoot(upd_treeview)
	Local tree_new:TGadget=AddTreeViewNode("NEW (0)",root)
	Local tree_upd:TGadget=AddTreeViewNode("UPDATES (0)",root)
	Local tree_curr:TGadget=AddTreeViewNode("CURRENT (0)",root)
	
	RedrawGadget updatepanel
	
	'If fSize <> -1 Then 
	  HTTPGetFile:Int(ini_update_server+PRP_UPD_LIST+".zip", PRP_UPD_LIST+".zip")
	'    Local zrObject:ZipReader = New ZipReader
		If ( zrObject.OpenZip(PRP_UPD_LIST+".zip") ) Then
			fileBuffer = zrObject.ExtractFile(PRP_UPD_LIST+".txt",False)
			'fileBufferRelease = zrObject.ExtractFile(PRP_CURR_LIST+".txt",False)
		      zrObject.CloseZip()
		   'Print "DLed..."+ini_update_server+PRP_UPD_LIST+".zip"
		End If
	
	If fileBuffer <> Null Then upd_list:String=LoadText(fileBuffer).split("~n") ' Get Stationfilenames from txt file
	
	If upd_list <> Null Then ' Tiny Failsafe if txt file is broken...   
	  
	    Local dir_stations:String[]=LoadDir( STATIONS_DIR,True)
	    Local fulldir:String
	    Local station_size:Int
	    Local station_size_str:String
	    Local sta:String=""
	    Local tmp:String=""
	    Local station_found:Int=0
	    Local stations_total_size:Int
	    Local stations_todo_size:Int
	    Local todo_size:String 
	
	    Local custom_station:Int=0
	
	
	For tmp=EachIn upd_list
	  tmp=tmp.Trim()
	  sta$=Get_StationName(tmp) 
	  fulldir$=STATIONS_DIR+sta$ 
	  station_size_str=Get_Filesize(tmp)
	  station_size=Int(station_size_str)
	  If FileType( fulldir$ )=1 Then
	   If station_size <> FileSize(fulldir$) Then      
	    ListAddLast download_list,tmp ' Move file to download list for later download and count size + filecount
	    maxcounter=maxcounter+1
	    stations_total_size:Int=stations_total_size+station_size
	   Else ' File is same, add to current list only
	    If Len(station_size_str) > 3 Then station_size_str=Left(station_size_str,Len(station_size_str)-3)+"."+Right(station_size_str,3)
	    AddTreeViewNode sta$,tree_curr, -1,station_size_str+" Bytes" 
	   EndIf 
	
	   Else 
	  ' file not present, needs to be downloaded!
	    ListAddLast download_list,tmp ' Move file to download list for later download and count size + filecount
	    maxcounter=maxcounter+1
	    stations_total_size:Int=stations_total_size+station_size
	  EndIf
	
	Next
	
	ModifyTreeViewNode (tree_curr, "CURRENT ("+CountTreeViewNodes(tree_curr)+")") ' update current treeview node with file numbers
	stations_todo_size:Int=stations_total_size ' set TO-DO-Size => Size of all files that need to be downloaded
	
	'DebugLog "Total size of all station files: "+stations_total_size
	'Print "Stations to download: "+maxcounter
	'Print "Total size of all stations: "+stations_total_size
	
	If maxcounter> 0 Then 
	
	Local FileArray:TStationFile[maxcounter] ' Static Index of all files that need to be downloaded 
	Local Index:Int = 0
	
	' Create Index entries from the Downloadlist
	For tmp = EachIn download_list
	    FileArray[Index]=New TSTATIONFILE
	    FileArray[Index].name = Get_StationName(tmp) 
	    FileArray[Index].size_str = Get_Filesize(tmp)
	    FileArray[Index].size = Int(FileArray[Index].size_str)
	    If Len(FileArray[Index].size_str) > 3 Then FileArray[Index].size_str = Left(FileArray[Index].size_str , Len(FileArray[Index].size_str) - 3) + "." + Right(FileArray[Index].size_str , 3) 'adding "."
	    'Print "added ["+Index+"] : "+FileArray[Index].name	 
	    If FileType( STATIONS_DIR + FileArray[Index].name ) = 1 Then FileArray[Index].ft=1 Else FileArray[Index].ft=0 
	    Index=Index+1
	Next
	
	' Check and dowload new stations
	'     SetGadgetText (label_upd1,"")
	     SetGadgetText (label_upd3,"0%")
	     SetGadgetText (label_upd2,"")
	     'SetGadgetText (label_upd2,GUI_LABELS_UPDATER[5]+"...")
	     SetGadgetText (label_upd4,"")
	     SetGadgetText (label_upd5,"")
	
	SetGadgetText (label_upd1,"Downloding Files... ") ' Set label to DL Stationlist
	RedrawGadget MainWindow
	
	Counter:Int = 0
	Index:Int = 0
	
	Local Progress_bak: Int = 0



	Local GetFile:TDataFetch2 = New TDataFetch2
	
	GetFile.Load(FileArray)
	
	
	While Not GetFile.isFinished()
		WaitEvent()
	
	      Select EventID()
	        Case EVENT_GADGETLOSTFOCUS
	            RedrawGadget (UpdatePanel)
	
	        Case EVENT_WINDOWCLOSE
	           End  
	
		  Case BUTTON_ACTION
	           Select EventSource()
	           Case Button_Update_back
	                End 
	        End Select
	
		Case EVENT_TIMERTICK

			GetFile.process(tree_new, tree_upd)
		
		
			Progress# = Float(maxcounter - GetFile.remaining)/Float(maxcounter)
			Progress_curr:Int=Int(Progress*100)
			
			If Progress_curr <> Progress_bak Then 
				SetGadgetText (label_upd3,progress_curr+"%")
				UpdateProgBar (upd_progbar , progress)
				RedrawGadget (UpdatePanel)
				Progress_bak=Progress_curr
			EndIf 
	
	    End Select
	
	'Until counter=maxcounter ' all files are downloaded or user hit EXIT button
	Wend

	GetFile.Free()
Print "FINISHED !!"
	
	
	EndIf ' maxcounter > 0
	    UpdateProgBar (upd_progbar,100)
	    SetGadgetText (label_upd3 , "100%")
	
	 	root:TGadget=TreeViewRoot(upd_treeview)
		tree_new:TGadget=AddTreeViewNode("NEW (0)",root)
		tree_upd:TGadget=AddTreeViewNode("UPDATES (0)",root)
		tree_curr:TGadget=AddTreeViewNode("CURRENT (0)",root)
	
	    SetGadgetText (label_upd1,"")
	    SetGadgetText (label_upd2,"")
	    SetGadgetText (label_upd4,"")
	    SetGadgetText (label_upd5,"")
	
	    SetGadgetToolTip(Button_Update_back,"Done!") ' Set tooltip
	    RedrawGadget (UpdatePanel)
	
	Else ' stationlist not found on server
	
	 SetGadgetText(label_upd1,PRP_UPD_LIST+".zip - File couldn't be found!") ' File Not Found Error MSG 
	
	EndIf 
	
	'Print "out of here" 
End Function 



Type TDataFetch2

	Const maxConnections:Int = 8
	
	Field curlMap:TMap = New TMap

	Field fileList:TList = New TList
	Field remaining:Int
	
	Field timer:TTimer
	
	Field finished:Int
	
	Field multi:TCurlMulti

	Method Load(fileArray:TSTATIONFILE[])

		' load the list
		For Local file:TSTATIONFILE = EachIn fileArray
			fileList.AddLast(file)
			remaining :+ 1
		Next
	
		Local connections:Int = Min(maxConnections, remaining)
	
		multi = TCurlMulti.Create()
	
		' load the curl handlers	
		For Local i:Int = 0 Until connections
			Local cstream:TCurlStream = New TCurlStream
			cstream.curl = TCurlEasy.Create()
			
			curlMap.Insert(cstream.curl, cstream)
			
			startFetch(cstream)
		Next
		
		' create a tick
		timer = CreateTimer(10)
	End Method

	Method process:Int(newTree:TGadget, updTree:TGadget)

		Local running:Int
		finished = True
	
		Local status:Int = multi.multiPerform(running)
	
		If running And status = CURLM_OK
			' wait for network 
			If multi.multiSelect() <> -1 Then
				' pull in any new data, or at least handle timeouts 
				status = multi.multiPerform(running)
				While status = CURLM_CALL_MULTI_PERFORM 
					status = multi.multiPerform(running)
				Wend
			End If
		End If
		
		If running Then
			finished = False
		End If
	
		Local messagesInQueue:Int
		
		' check for finished curls
		Local msg:TCurlMultiMsg = multi.multiInfoRead(messagesInQueue)

		While msg
		
			Local cstream:TCurlStream = TCurlStream(curlMap.ValueForKey(msg.easy))
			cstream.stream.Close()

			' bit messy to have this here.. oh well :-/
			If cstream.file.ft = 1 Then
				AddTreeViewNode cstream.file.name , updTree , - 1 , cstream.file.size_str + " Bytes"
				ModifyTreeViewNode (updTree, "UPDATES ("+CountTreeViewNodes(updTree)+")") 
			Else
				AddTreeViewNode cstream.file.name, newTree, -1, cstream.file.size_str+" Bytes"
				ModifyTreeViewNode (newTree, "NEW ("+CountTreeViewNodes(newTree)+")") 
			EndIf 

		
			' remove the easy handle
			multi.multiRemove(msg.easy)
			
			' get the next file
			If remaining > 0 Then
				running = True
				startFetch(cstream)
				
				finished = False
			End If
			
			msg = multi.multiInfoRead(messagesInQueue)
		Wend

		Return running
	End Method
	
	Method startFetch(cstream:TCurlStream)
		Local file:TSTATIONFILE = TSTATIONFILE(fileList.RemoveFirst())

		If file Then
			remaining :-1
		
			Local dest:String = STATIONS_DIR + file.name
			
			Local url:String = ini_update_server+"stations/" + file.name
	        'url = url.Replace("http://", "http::")

			cstream.file = file
			cstream.stream = WriteStream(dest)
	
			cstream.curl.setOptString(CURLOPT_URL, url)
			cstream.curl.setWriteStream(cstream.stream)
			cstream.curl.setOptInt(CURLOPT_FOLLOWLOCATION, 1)
			
			' add to the multi handler
			multi.multiAdd(cstream.curl)
		End If
	End Method
	
	Method isFinished:Int()
		Return finished
	End Method
	
	Method Free()
		StopTimer(timer)
		
		For Local cstream:TCurlStream = EachIn curlMap.Values()
			cstream.curl.cleanup()
		Next
		
		If multi Then
			multi.multiCleanup()
		End If
	End Method

End Type

Type TCurlStream

	Field stream:TStream
	Field curl:TCurlEasy
	Field file:TSTATIONFILE

End Type

Oh, it runs fine non-threaded. I haven't tested it while running threaded, but it should work the same for both.

Last edited 2013

Brucey

(Posted 2013) [#43]

I think it is a reasonably good example of how 3rd party libraries can be used to prevent the need to re-invent the wheel so often.

I always find it interesting how many wheels I see re-invented on the forums. Of course, I understand how "cool" it is to be able to write something completely in BlitzMax code, but often 3rd party libraries can do the same work better and faster than pure BlitzMax code.

Still, choice is also a good thing, so :-)

Grisu

(Posted 2013) [#44]

Thanks! That helped a lot. If you don't know it better than you'll have to reinvent the wheel. ;)

In my first tests I recognised that your routine will create empty prp-files when you interrupt the download process. Therefore I'll add some extra code to delete such files.

I did the treeview list so that users can recheck what changes were made and directly run a new / updated radio station from there. In order to optimise the code a bit I think I can get rid of the Filearray. As most stuff needed is already in the "download list".

Did you change much in the libidn (dev) version. I downloaded the full SVN list of modules but it's not in there.

It's great that might not need to run threaded. As I found out BMX uses more memory for this and the garbage collector seems to act slower.

In addition I might use the lib for file loading (from HD) as well, which might give the whole app a small speed boost.

But first things first: I need to test my full code on all platforms. This will take a while. Too bad this weekend is nearly over.

Derron

(Posted 2013) [#45]

@Brucey: if using only blitzmax, you only have to rely on its internals. If using your libcurl, you have to install mingw, have to assure your lib working etc.

Yes libcurl is a really round wheel with not much work to be done for getting it doing what you want.

@grisu: I coded my example using Linux Mint 13 64Bit - and it worked fine. Replace "WaitEvent" with "PollEvent" and it will hang less between the downloads - and it takes away the need of the timer.

bye
Ron

edit: take also into consideration that the used method was a kind of idea making ressource loaders thread friendly... but as events aren't you should not care that much (except you alter the event system to use mutexes in the case of an threaded compilation).

Last edited 2013

Brucey

(Posted 2013) [#46]

Did you change much in the libidn (dev) version

I only mentioned libidn because the header was missing when I tried to build on Linux, so I needed to install the libidn dev package. The library itself is usually installed on Linux by default.

if using only blitzmax, you only have to rely on its internals

Indeed, but since BlitzMax internals are lightweight to say the least, I think it's better to use the right tool for the right job.
And if you are a programmer, using MinGW and third-party libraries should not be a wall against which you cannot climb (or knock down).
For someone who says "Well, if it doesn't come with BlitzMax I'm not using it" , I feel rather sorry for, as if they want to hide in their shell and never see what the real world can offer them, then it's really too bad, and their software might be worse off for it.

I tested my example using Linux Kubuntu 64-bit (headless) - and it also worked fine ;-)

Grisu

(Posted 2013) [#47]

Solve one problem get another: GetFile.Free() isn't properly deleting all open file handles if one aborts the current download. MaxConnections-1 empty files stay inside the stations folder.

I thought it would be easy to use deletefile() inside the Method Free(), but the files seem to be "protected" (still in use?). Thus deletefile() fails.

So I tried to delete them in the main program after exiting the loop, by going through the stations folder again and detect all files with filesize=0. The accoding files still can't be erased. Deletfile() returns = 0.

I can delete them by hand though, but this is no valid option... ;) - Any ideas on that?

On the bright side:
The code works fine under my Ubuntu 12.10. You can grab an updated full distribution test package from below:

Linux Download: http://www.mediafire.com/?7rzethd7jbdmdpv

Last edited 2013

Brucey

(Posted 2013) [#48]

The streams will need to be closed first before you can delete the files. And before that, probably to shut down the curl connections.

The best thing to do will be to close the multi handle, then close all of the easy handles, then close all of the streams...

Grisu

(Posted 2013) [#49]

These changes work fine... Testing...

	Method Free()
		StopTimer(timer)

		If multi Then
			multi.multiCleanup()
		End If
		
		For Local cstream:TCurlStream = EachIn curlMap.Values()
			cstream.curl.Cleanup()
			cstream.stream.Close()
			If FileSize (STATIONS_DIR + cstream.file.name) = 0 Then
			  DeleteFile(STATIONS_DIR+cstream.file.name)
			 ' Print STATIONS_DIR + cstream.file.name
                 'Else
                   'Print "ok - not to be deleted"
			EndIf 
		Next		
	End Method

Last edited 2013

Brucey

(Posted 2013) [#50]

Probably better to cleanup the curl first before you close the stream.

Grisu

(Posted 2013) [#51]

Done and added another life line.

|Edit| The Linux test package has been updated accordingly.

Last edited 2013