[darcs-devel] darcs Windows <-> Linux woes, and recommendations

Dave Roberts ldave at droberts.com
Mon Feb 21 08:39:50 PST 2005


Sorry about this, folks. I had sent this a couple of days ago and it had
bounced back to me. I forwarded it to David Roundy who kindly pointed
out that I had forwarded it to the darcs-devel-request admin address by
mistake. Of course, the bot took one look at it, rejected it all, and
send it back to me with a spanking attached. Anyway, I think I clicked
on the wrong address in the subscription confirmation email I received.
I apologize for the delay and confusion.

As I said on Saturday, not five minutes after I sent this, Benedikt
Schmidt seems to have supplied a patch for exactly what I was wanting,
but there is still some value in reading through the experience.

Finally, I would also make a third recommendation that isn't listed
here: darcs needs to improve its error messages. Darcs has a wonderful
simplicity in its command set and conceptual workflow. Unfortunately,
when darcs encounters a problem, it often reports it in vague, and
sometimes misleading ways. When I was debugging some of the stuff below,
I was often helped more by the subprograms which were being called than
by darcs itself. Darcs would frequently report things like "invalid
repository" instead of "DARCS_SFTP reports 'invalid password'" or
something like that. The key thing here is to provide users clues that
allow them to correct the underlying problem. I know that this is
particularly difficult when subprograms are involved because of the
linkages between the two, but this is also where it is most often
needed.

Cheers,

-- Dave

-------- Forwarded Message --------
From: Dave Roberts <ldave at droberts.com>
To: darcs-devel-request at darcs.net
Subject: darcs Windows <-> Linux woes, and recommendations
Date: Sat, 19 Feb 2005 10:40:14 -0800
Dear darcs developer gurus,

I have just spent the last couple of weeks getting darcs working between
my Windows and Linux systems. I would like to share my experiences with
you. As you might expect, I have some recommendations for enhancements.

First, let me say that darcs works well on both systems in isolation. I
was able to install binaries using Yum for Fedora Core 3 and a simple
zip file for Windows. On the individual systems, I used darcs to perform
various operations and everything seems to work fine; simple and easy.

My trouble started when I wanted to get/push/pull between repositories
on the two operating systems. In my case, my "master" repository is on
the Linux side and I simply want to be able to grab it with my Windows
laptop, go offline for a while, make a bunch of changes, and then push
those back to the Linux side.

Darcs supports a few different options:

1. Use NFS or CIFS/Samba to share filesystems. I didn't try this, mostly
because I want to be able to share patches over the Internet when I'm
working remotely and neither of these protocols is really suited to
wide-area scales. In my case, my Windows machine is my laptop and my
Linux box sits in my home office, behind a cable-modem-attached
firewall.

2. I could use http or ftp. These worked well for gets and pulls, but
doesn't for pushes. I could set up an http or ftp server on my Windows
laptop and pull from my Linux box, but this can be difficult when I'm
working from a remote location, possibly with the laptop behind an
unknown firewall, etc. Not to mention the pain of configuring all those
servers on a Windows laptop. In this case, I really want to be able to
make contact with the more stationary machine and push changes to it.

3. After some confusion in the manual about the capabilities of http/ftp
transport, people on the darcs-users list recommended ssh for pushing
changes to the host. The most popular SSH implementation on Windows is
Putty. I already had Putty installed and so I started to configure
things. The docs on SSH transport weren't very clear, either, so I
bumped around doing that for a while. I finally got push working with
Putty.

The big issue here is that when performing a get or pull, darcs calls
SFTP in a way that Putty doesn't quite support, leading to an error. The
net result is that SSH can't be used for get/pull, only for push. This,
of course, is terribly inconvenient if you're trying to get things
configured correctly. You now have to get both a web or ftp server set
up, as well as ssh configured correctly. Further, when dealing with
firewalls, etc., you either have to make multiple holes or add
complexity by doing something like tunneling http over ssh, etc.
Finally, you have to type in the repo each time, depending on what
you're doing. When doing get or pull, you use

darcs pull ftp://user:password@host/repo

When pushing, you then switch to

darcs push user at host:repo

where your password is handled by ssh public keys.

All in all, very complex for no good reason other than working around
limitations.

So, last night, I did some digging, trying to figure out what could be
changed.

The root issue of darcs/psftp incompatibility seems to stem from the way
that darcs invokes psftp when downloading all the patches during a get
or pull. Darcs seems to convert:

darcs get user at host:repo

into a call to DARCS_SFTP

$DARCS_SFTP -b /temppath/tempfile user at host:repo/_darcs/patches

where tempfile contains a list of sftp get commands, one for each patch
that is downloaded.

In Open SSH, the user at host:repo/_darcs/patches form causes sftp to
execute a cd command to repo/_darcs/patches from whatever the login
directory is.

The problem is, Putty's psftp only recognizes a user at host form and seems
to ignore everything after the colon. Thus, after psftp connects, the
session is left with the cwd as the login directory. All the get
commands cooked up by darcs in the temp file are of the form "get
<patchname>.gz" and they fail.

Now, there are a couple of options:

1. The easiest thing would be to contact the author of Putty and ask
that he please implement command line support for psftp identical to
that provided by OpenSSH. That, of course, is putting your functionality
in the hands of somebody else. I think we can do better.

1a. You might be tempted to say, "Use another ssh implementation on
Windows." Trouble is there aren't too many. There are a couple that use
a port of Open SSH, but they don't seem well supported. One is dependent
on Cygwin (yet another dependency!) and another seems to have limited
support. In short, Putty is the best option and most popular ssh
implementation on Windows.

2. We could change darcs to invoke the DARCS_SFTP program with:
$DARCS_SFTP -b /temppath/tempfile user at host

where the first command in tempfile is "cd repo/_darcs/patches",
followed by all the gets that would normally be there. It looks like
some rework of the copySSHs function in External.hs would do the trick.
Trouble is, I'm no Haskell hacker and I don't even play one on TV.

With this one change, I think you would have get/pull working over ssh
on both Linux and Windows.

So that's the first recommendation.

A second recommendation it that darcs *needs* some debugging command
line flags that allow users to see when and how subprograms are being
invoked. Right now, there is no way to see how a program is actually
being called. I would add a --debug option to every command to cause it
to spit out all sorts of diagnostic information as it runs, including
information about subprogram invocations, where it is storing temp
files, etc., etc. I would not augment --verbose as it gets turned on for
general usage and you don't want all that heavy diagnostic output
cluttering an already verbose output.

So, that's it. Thanks for creating a great program. When using it on a
single host, it's simple and streamlined. A few changes would make the
Windows remote usage almost as easy as single machine usage.

-- 
Dave Roberts <ldave at droberts.com>





More information about the darcs-devel mailing list