Hi. I have found a really hard to find bug that is very consistently repeatable. Before getting into the details, I think it should be noted that this issue will be resolved if the Renci.SshNet.SshClient.Dispose() and the Renci.SshNet.ForwardedPortLocal.Dispose() (and anything else) actually and honestly dispose everything. But the problem is that after calling those, there are still threads hanging around doing stuff and sometimes they get stuck badly. It looks like someone went crazy with anonymous methods and launching Thread Tasks without leaving a handle to the Thread for management by the parents to kill them when disposed. And I found in Session.NET40.cs this line: Task.Factory.StartNew(action, TaskCreationOptions.LongRunning);which I changed to: Task.Factory.StartNew(action, TaskCreationOptions.AttachedToParent);but it had no positive impact.I have seen posts on this site about Disconnect doing funny stuff and also about the Timeouts not seeming to disconnect things, and some other things like that. I bet they are related to getting over your head in multi-threading or events.The result is that sometimes I still have the process listening on the local port way after the Dispose() is called. And also the Connection is sitting there too. Usually this happens when eliminating both the Forwards and the Connection, but sometimes even just the Forwards. The call to the Forward's Disconnect() and Dispose() when the problem happens will take a much longer time, but then sadly continue going after seemingly giving up.When I pause I can see 2 Renci threads hanging around and stuck at these points:ForwardedPortLocal.NET.cs line 34: var socket = this._listener.AcceptSocket();Session.cs line 1575: var message = this.ReceiveMessage();I have tested this A LOT!!! trying to find ways around it. It happens in the code base (pulled today 12/5/12), it happens in .Net 4.0 DLL from a month ago, and it happens from .Net 3.5 DLL pulled today also. Another result is that sometimes the situation causes the app to die a horrible death with:System.AggregateException was unhandled Message=A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. Source=mscorlib StackTrace: at System.Threading.Tasks.TaskExceptionHolder.Finalize() InnerException: System.NullReferenceException Message=Object reference not set to an instance of an object. Source=Renci.SshNet StackTrace: at Renci.SshNet.ForwardedPortLocal.b__0() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.Execute()WHEN DOES THIS HAPPEN:Here is the kicker. I wrote an app to manage SSH tunnels. And when I first was having problems with Renci, I made it so I can implement other libraries and other means. Via a config file I can add and remove tunnels and have multiple connections. The process reads the config file periodically and applies the changes found. So I can have a connection using Renci, another using Chilkat, another using Plink, Tunnelier, and that's all I finished implementing so far.If I have all Renci connections, it seems that I can add and remove tunnels and connections a bunch of times and not have an issue. (however usually there is a Connection (SshClient) that hangs around in CLOSE_WAIT state). If I bring say Chilkat into the mix, things still are OK.. and I can replace all active connections to one of the other implementations.The problems start happening when I bring in Plink or Tunnelier. They are launched using System.Process. They close themselves properly from what I see in memory and via TCP connections and SSH server. When I have a Renci and Plink (let's say) connection, it is then that if I start removing tunnels from Renci I might have issues. If I start removing the Connection, I almost always have an issue.The issue is that when Renci is told to close up shop, it stalls for a while, and then the code moves on while leaving the listeners in place (though of course not working). And they stay there it seems indefinitely. And I'm also using System.Net.NetworkInformation.IPGlobalProperties.GetIPGlobalProperties().GetActiveTcpListeners() to look at the listeners and it's there. It's because there are those silly Renci threads that are hanging around for some reason trying to read from the socket.And I have verified that the Connection replacement (like Plink) is not actually opening up that tunnel. I have my app now specifically checks for the Active listeners before moving tunnels to another connection.This may be a very complicated specific bug to find, but I think the real root of the problem and the much easier one to fix, is that when you Dispose(), it really should be DISPOSED, leaving no threads behind. I have also verified that calling SshClient.Dispose() does not DISCONNECT, so you have to call both. I think also similar thing with the Forward.. if you don't call Stop().This is how I stop the connections and port forwards: foreach (ForwardedPort forward in this.sshConnection.ForwardedPorts.ToList()) { this.sshConnection.RemoveForwardedPort(forward); Console.WriteLine("just finished removing tunnel"); } this.sshConnection.Disconnect(); Console.WriteLine("just finished conn Disconnect()"); this.sshConnection.Dispose(); Console.WriteLine("just finished conn Dispose()"); public void Dispose() { if (this.localForward != null) { this.localForward.Stop(); Console.WriteLine("just finished tunnel Stop()"); this.localForward.Dispose(); Console.WriteLine("just finished tunnel Disconnect()"); //System.Threading.Thread.Sleep(2000); } }
Comments: ** Comment from web user: olegkap **
Comments: ** Comment from web user: olegkap **
Hi,
Can you give it another try?
I just committed 23481 code where I fixed 2 possible issues for leaving thread running in port forwarding scenario.
Can you try it again and see if it was fixed in your situation?
Thanks,
Oleg