[darcs-devel] http pipelining

David Roundy droundy at darcs.net
Mon Dec 17 16:15:19 UTC 2007


On Mon, Dec 17, 2007 at 10:36:08AM -0500, Tim Chevalier wrote:
> On 12/17/07, David Roundy <droundy at darcs.net> wrote:
> >
> > Any suggestions how to go about doing this? I quick look suggests that
> > libcurl can't handle http pipelining, and that no haskell HTTP library does
> > so.  The only library I can find that does seem to support it is libwww,
> > which looks like a real pain to use.
> 
> I spent a few hours trying to implement this in darcs a few months
> ago, by using multiple Haskell threads in combination with
> Network.HTTP to simulate pipelining. I don't remember where I got
> stuck anymore, and I don't even know if that approach makes any sense
> to begin with.

I saw this on the bug tracker.  Any chance you'd be interested in taking
another look at it? At a minimum, the same infrastructure would be needed,
in that we would in either case need to figure out how to spark downloads
in parallel.  I think some sort of "par" approach to parallelization of
pure code ought to be able to handle this, as almost all our http requests
are wrapped in unsafeInterleaveIO wrappers (which should actually be safe,
since we always download straight to disk, and *when* we download a given
file should never matter... nor should it matter if we download them
needlessly).
-- 
David Roundy
Department of Physics
Oregon State University


More information about the darcs-devel mailing list