[darcs-users] ssh issues (and Windows too)

[Eric Kow]

Hi all,

Happy New Year!

Just dumping some notes from some recent work on the Darcs SSH code.  The patches here are either on http://darcs.net or on the tracker.

Bug-fixes: these may be bugs that only affect people using bleeding edge darcs or betas

1. ssh performance regression (patch686) - darcs 2 introduced a darcs transfer-mode command which lets us share connections to ssh servers.  Unfortunately, sometime last year, we accidentally introduced an error which made darcs not realise when it could use the darcs transfer-mode connection it had previously opened.  The effect is that ssh would slow down because for each file darcs wanted to fetch, it would call darcs transfer-mode anew.

2. scp error (patch689) - if darcs transfer-mode failed, darcs falls back to scp.  Lately, it'd started saying "scp example.com example.com:foo bar instead of "scp example.com:foo bar"

UI improvements and documentation :

3. the one minute Windows/SSH guide - for now http://wiki.darcs.net/Windows - problems with SSH on Windows seem to be quite frequent, so maybe a prominent guide would help?

4. default to plink/pscp/psftp on Windows (patch687) - this should bring darcs a tiny bit closer to just working on Windows.  Some caveats, though, see #3

5. relay ssh stderr (patch688) - this should fix the longstanding issue845 where darcs appears to hang on Windows when ssh'ing because actually ssh wants to ask you things and darcs isn't showing you.  But there's a pretty massive BUT (see #7)

Open questions:

6. can we dump the ControlMaster feature? (patch690) - I keep itching to do this because it feels like the extra code complexity is just a liability.  Comments?

7. darcs transfer-mode on Windows :-( - OK, the good news is that if you follow the guide in #3, darcs transfer-mode *should* work on Windows, even with PuTTY. Unfortunately, we've got a tricky problem interacting with putty which makes it look like we haven't really fixed the issue in #5.

I'm not sure what the real problem is, but the symptom is that darcs appears to hang when trying to communicate with SSH hosted repositories.  One potential cause of the problem is that PuTTY SSH seems to talk to users by reading from stdin and writing to stderr, whereas OpenSSH reads from and writes directly to the terminal device.  I at least know this to be true because plink foo.com > NUL 2> NUL shows no output, whereas ssh foo.com > NUL 2> NUL still shows it.  I'm not sure that's the root cause of the problem though because it also hangs on OpenSSH on Windows.  So maybe there are two things going on.

(NB: developer-centric stuff below, maybe worth skipping)

But focusing on this first problem, I think I'm a bit in over my head to be able to fix this.  This is where we start getting into implementation details.  If I understand correctly, the basic set up is that we call "ssh/plink example.com darcs transfer-mode --repodir /foo" with runInteractiveProcess.  This gives 3 Handles that we can use to put stuff into stdin, and read from the stdout/stderr as seen by plink/ssh.

Easy, right? Just start dumping the password into the stdin handle, and printing the stderr handle output.  Unfortunately, it's a bit trickier than that, because we actually use these handles to communicate with the remote darcs (via the ssh process).  For example, when darcs transfer-mode starts up, we read the line "Hello  user, I am darcs transfer mode" from the ssh output.  Whenever we want to copy and file, we write to the stdin handle instructions for what file to fetch and then read the contents from stdout.  If something goes wrong, we report the stderr output.

So where I'm stuck is how to avoid mixing up the bits where talking to the ssh process is really about talking ssh itself, and the bits where the intention is to talk to the darcs on the other end of the line.  I thought maybe we could do something like use the fact that we've read "Hello user, I am darcs transfer mode" that we're shifting to talk-to-the-other-side time.  But I suspect I'd be just setting us up for some kind of crazy concurrency issue, or things related to blocking on handles.  I don't really know how to approach this.  Hopefully it's way easier than it looks and I'm just making a big deal out of nothing.  

Comments? Laterally-thought solutions [like I wonder if changing PuTTY would be feasible, or getting people to use something else]?

-- 
Eric Kow <http://erickow.com>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 163 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20120101/b7321f8c/attachment.asc>