[darcs-devel] ssh ControlMaster debugging

[Alexander Staubo]

On 10/21/07, Eric Y. Kow <eric.kow at gmail.com> wrote:
> Alexander,
>
> > It would make sense build an isolated test case for this, in Haskell.
> > Something that set up ssh in control master mode and repeatedly
> > invoked scp and/or sftp.
>
> That is a very nice idea.  The tester is attached.  You'll have to put
> it in the src directory of darcs and add the following to GNUmakefile

Great work. It seems to hang randomly here: 78, 16, 5, 81, 151, etc.

On the client side, scp is hanging in read(). On the remote side,
there's a "notty" SSH process sitting in select() on two file handles,
one being the connection itself, the other being a FIFO pipe opened
for reading. So they're not talking.

At the point of hanging the SSH server's debug log ends with:

debug1: session_by_pid: pid 27552
debug1: session_exit_message: session 0 channel 0 pid 27552
debug1: session_exit_message: release channel 0
debug1: session_by_channel: session 0 channel 0
debug1: session_close_by_channel: channel 0 child 0
debug1: session_close: session 0 pid 0
debug1: channel 0: free: server-session, nchannels 1

This is consistent with how a normal session finishes; ie., at this
point, the session is terminated, although the connection is still
alive.

I don't know anything about SSH's internals, but think the SSH server
expects the client to disconnect at this point, because a normal
connection -- one where the client side does not hang -- terminates
this way:

Connection closed by 194.63.250.55
debug1: do_cleanup
debug1: PAM: cleanup
Closing connection to 194.63.250.55
debug1: PAM: cleanup

I will need to compile OpenSSH with debug symbols enabled in order to
get a sensible scp stack trace from gdb.

Alexander.