[darcs-users] GSoC Project: Improving darcs' network performance

Eric Kow kowey at darcs.net
Fri Apr 9 17:55:48 UTC 2010

On Fri, Apr 09, 2010 at 10:25:52 -0700, Jason Dagit wrote:
> >  * What happened to HTTP pipelining? Why wasn't that successful? Is
> > problem in darcs, or in libs, or somewhere else? (Or nobody knows?) It
> > would be worth to write about it, as it intended to solve the same
> > problem as my project.
> >
> Darcs supports HTTP pipelining.  If you're not seeing the pipelining
> happening, then I'm not sure what is wrong.

Perhaps a better question to ask is how this project relates to HTTP
pipelining.  Why wasn't Darcs's use of pipelining successful enough
that it would be a good idea to implement a smart server?

Some of our experiences with pipelining:

 - flakiness (cf. spate of bugs of curl_multi_perform bugs)
 - seemingly non-universal support (but then, how would a smart
   server help things?)

> The benchmarks on the wiki are really hard to find because they seem to be
> strewn across many small pages, but a few links might get you started.
> Pages about benchmarks:
> http://wiki.darcs.net/Development/Benchmarks
> http://wiki.darcs.net/Benchmarks

The second page you linked <http://wiki.darcs.net/Benchmarks> is the
one-stop shop for benchmark results.  The first page is mostly for
developing the benchmark infrastructure.  Hmm, any ways we could improve
things so that the results are easier to find?

> And then there is a slew of individual runs on machines like this one:
> http://wiki.darcs.net/Benchmarks/Quasar

Yes, this is by design (partly to avoid having one giant page).

Note that each of the machine-specific pages is expected to follow
a general template, although there are bits and bobs scattered

Now the bad news: we still don't have any benchmarking for the network
stuff, partly because we haven't yet wrapped our heads around how to
go about it in principle.
> >  * In description of 'darcs optimize --http' command, I'm saying that
> > patches before tag don't get reordered. Actually, I'm not 100% sure
> > about that. The idea I got while looking at sources is that tag is
> > regular patch (to some degree) that doesn't commute with anything.
> > However, I didn't find prove of it in sources, so I may be wrong. The
> > technical explanation of what's really going on with tags would be
> > really helpful.
> >
> I think for most cases this is accurate.  A tag depends on all the patches
> before it so most repository interactions stop at tags.  It's possible
> somethings need to look beyond tags, but I don't know the details.

We want to be careful here, since (as I understand it) a tag is just an
empty patch with explicit deps on a set of patches (including prior
tags).  So this means that you can commute things behind tags, and you
can have tags on top of patches...

Say you have (where T1 depends on TP1 TP2)

  TP1 TP2 T1 O3

While it's true that T1 will never get pushed past TP2 (and vice-versa),
you could always have situations that look like

  TP1' O3 TP2' T1

OK, so that stuff we all know, but surely we can still say useful things
about the sort of guarantees Darcs offers on how tagged stuff is stored.

Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
PGP Key ID: 08AC04F9
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20100409/ef707c6c/attachment.pgp>

More information about the darcs-users mailing list