Libcurl and corrupt files

BlitzMax Forums/Brucey's Modules/Libcurl and corrupt files

Volker

(Posted 2010) [#1]

Hi,

I'm using libcurl to upload zip files to ftp.
But every 3rd or 4th uploaded archiv is corrupt.
Libcurls debug function is reporting that
binary mode is used, not ASCII.

Every hint what to search for is appreciated.
Are there any parameters to play with?

The code is mostly from the examples.


Function FTPUploadFileMulti(HostName:String, filename:String, directory:String, username:String, password:String)

	Local multi:TCurlMulti = TCurlMulti.Create()
	Local curl:TCurlEasy = multi.neweasy()
	
	Local urls:String[]
	urls = [HostName + "/" + directory + "/" + StripDir(filename) ]
	
	' Just here for keeping an eye on things
	curl.setOptInt(CURLOPT_VERBOSE, 1)
	curl.setDebugCallback(debugFunction)
	' Create our upload-helper :-)
	Local upload:UploadHelper = UploadHelper.Create(filename)

	' ftp connect and initial directory
	curl.setOptString(CURLOPT_URL, urls[0]) ' "<full FTP URL including filename To upload>")
	
	
	curl.setOptString(CURLOPT_USERPWD, username + ":" + password)
	curl.setOptInt(CURLOPT_UPLOAD, True)
	curl.setOptInt(CURLOPT_FOLLOWLOCATION, 1) ' follow redirects
	curl.setOptInt(CURLOPT_MAXREDIRS, 2) ' maximum redirects
	curl.setOptInt(CURLOPT_CONNECTTIMEOUT, 100) ' timeout '15
	' set the upload size
	curl.setOptLong(CURLOPT_INFILESIZE_LARGE, upload.size)
	
	' set the callback
	curl.setReadCallback(upload.Read, upload)

	Local running:Int
	' do the UPLOAD !
	Local status:Int = multi.multiPerform(running)

	While status = CURLM_CALL_MULTI_PERFORM
		status = multi.multiPerform(running)
	Wend

	While running And status = CURLM_OK
		' wait for network 
		If multi.multiSelect() <> - 1 Then
			' pull in any new data, or at least handle timeouts 
			status = multi.multiPerform(running)
			While status = CURLM_CALL_MULTI_PERFORM
				status = multi.multiPerform(running)
			Wend
		End If
	Wend

	multi.multicleanup()

End Function

Function debugFunction:Int(data:Object, msgType:Int, message:String)

	' we only care about server info
	If msgType = CURLINFO_HEADER_IN Then
		Print message
	End If

End Function

' This just processes a filestream for us.
' DoRead, does the hard work of feeding the callback buffer with data from the stream.
Type UploadHelper

	Field name:String
	Field size:Int
	Field currentPos:Int
	
	Field stream:TStream

	Function Create:UploadHelper(file:String)
		Local this:UploadHelper = New UploadHelper
		
		this.name = file
		this.size = FileSize(file)
		
		Return this
	End Function
	
	Method DoRead:Int(buffer:Byte Ptr, size:Int)
		If Not stream Then
			stream = ReadStream(name)
			currentPos = 0
		End If
		
		Local count:Int = Stream.Read(buffer, size)
		currentPos:+count
		
		Return count
	End Method

	Function Read:Int(buffer:Byte Ptr, size:Int, data:Object)
		Return UploadHelper(data).DoRead(buffer, size)
	End Function

End Type

Volker

(Posted 2010) [#2]

Looks like the failure only appears on certain servers.
Will do some more investigations.

Htbaa

(Posted 2011) [#3]

Hey Volker,

Did you get error message 18?

CURLE_PARTIAL_FILE (18)

A file transfer was shorter or larger than expected. This happens when the server first reports an expected transfer size, and then delivers data that doesn't match the previously given size.

It's an error I'm getting when doing HEAD requests. Was wondering if this is related.

Edit: Never mind, my problem lies elsewhere. cURL complains when a HEAD request reports a Content-Length but doesn't supply the content body. Which is strange as HEAD requests aren't supposed to give a content body but should report the Content-Length on forehand.

Last edited 2011