Quantcast
Channel: sshnet Issue Tracker Rss Feed
Viewing all articles
Browse latest Browse all 1026

Created Unassigned: Upload progress is reported in random order [2463]

$
0
0
UploadFile() Method in SftpClient has an optional uploadCallback parameter which can be used for tracking upload progress. The problem is, each invocation may arrive on a different thread, in any order, because internally this is called:

```
this.ExecuteThread(() => uploadCallback(writtenBytes));
```

This is problematic - I am calculating progress based on the total size of a file, but get progress messages in random order (5%, 15%, 10%...). Additionally, I was logging to a file, and used 100% progress callback to close the output stream, which was crashing my application (as there were more progress updates "from the past" to come).

I fixed it locally by just reporting progress on the upload thread:

// Inside InternalUploadFile
```
try
{
uploadCallback( writtenBytes );
}
catch( Exception e )
{
// Don't let the user code break the upload loop
System.Diagnostics.Trace.TraceError( "UploadCallback error: " + e.ToString() );
}
```

I think that calling progress callback on a thread-pool thread is an overkill and this simple solution is acceptable, but if it really has to be asynchronous, then perhaps solution similar to this would work better:

Helper class:

```
internal class ProgressUpdater
{
private readonly Action<ulong> m_uploadCallback;
private readonly ManualResetEvent m_waitable = new ManualResetEvent( false );
private readonly BlockingCollection<ulong?> m_progressEntries = new BlockingCollection<ulong?>();

public ProgressUpdater( Action<ulong> uploadCallback )
{
Debug.Assert( uploadCallback != null );
m_uploadCallback = uploadCallback;
ThreadPool.QueueUserWorkItem( DoWork );
}

internal void ReportProgressAsync( ulong progress )
{
m_progressEntries.Add( progress );
}

internal void Finish()
{
m_progressEntries.Add( null );
m_waitable.WaitOne();
m_progressEntries.CompleteAdding();
}

private void DoWork(object state)
{
while( true )
{
ulong? progress = m_progressEntries.Take();
if( progress.HasValue )
{
try
{
if( m_uploadCallback != null )
{
m_uploadCallback( progress.Value );
}
}
catch( Exception e )
{
// Do not allow user-injected progress handling code crash the application -
// This is otherwise likely, as this exception is now on a thread-pool thread,
// and is unhandled.
Trace.TraceError( "UploadCallback exception: {0}{1}Details:{1}{2}", e.Message, Environment.NewLine, e.ToString() );
}
}
else
{
// Finish
m_waitable.Set();
break;
}
}
}
}
```

And in the InternalUploadFile():

```
// ... some code omitted
var progressUpdater = new ProgressUpdater( uploadCallback );
do
{
// Cancel upload
if (asyncResult != null && asyncResult.IsUploadCanceled)
break;

if (bytesRead > 0)
{
if (bytesRead < buffer.Length)
{
// Replace buffer for last chunk of data
var data = new byte[bytesRead];
Buffer.BlockCopy(buffer, 0, data, 0, bytesRead);
buffer = data;
}

var writtenBytes = offset + (ulong)buffer.Length;
this._sftpSession.RequestWrite(handle, offset, buffer, null, s =>
{
if (s.StatusCode == StatusCodes.Ok)
{
Interlocked.Decrement(ref expectedResponses);
responseReceivedWaitHandle.Set();

// Call callback to report number of bytes written
progressUpdater.ReportProgressAsync( writtenBytes );
}
});
Interlocked.Increment(ref expectedResponses);

offset += (uint)bytesRead;

bytesRead = input.Read(buffer, 0, buffer.Length);
}
else if (expectedResponses > 0)
{
// Wait for expectedResponses to change
this._sftpSession.WaitOnHandle(responseReceivedWaitHandle, this.OperationTimeout);
}
} while (expectedResponses > 0 || bytesRead > 0);

this._sftpSession.RequestClose(handle);

progressUpdater.Finish();
```

This way all progress messages are queued and processed serially, by the same thread-pool thread.

Viewing all articles
Browse latest Browse all 1026

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>