[darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

Dmitry Kurochkin dmitry.kurochkin at gmail.com
Fri Dec 21 01:12:49 UTC 2007


I have completed initial work on libwww pipelining. Output of darcs whatsnew
is attached (sorry for that, I will try to make a proper patch tomorrow).
What is done:
- libcurl functionality is implemented using libwww. Now pipelining works.
- New Libcurl module provides 3 functions:
  * copyUrl - same as copyUrl from Curl.hs. It uses copyUrls and waitNextUrl.
  * copyUrls - takes (filename, url) list, creates requests and adds
them to libwww. Does not load anything.
  * waitNextUrl - starts libwww event loop and blocks until first url
loads (or error happens). After it returns it should be possible to
add more urls to queue using copyUrls again. waitNextUrl should be
called as many times as urls are in the queue.

At the moment the only place where copyUrls is used is get command.
But I hope this interface
is enough for Darcs. If not - we need to think of smth more complex.
Waiting for comments here.

What is missing:
- DARCS_PROXYUSERPWD is not used (but http_proxy works).
- Proper error handling.
- Not tested.
- ???

Performance:
- libwww with pipelining disabled:

> time ~/projects/darcs/darcs get http://darcs.net/repos/unstable
Copying patch 4847 of 4847... done.
Applying patch 4847 of 4847... done.
Finished getting.
~/projects/darcs/darcs get http://darcs.net/repos/unstable  23,95s
user 7,54s system 1% cpu 27:15,17 total

- libwww with pipelining enabled:

> time ~/projects/darcs/darcs get http://darcs.net/repos/unstable
Copying patch 4847 of 4847... done.
Applying patch 4847 of 4847... done.
Finished getting.
~/projects/darcs/darcs get http://darcs.net/repos/unstable  71,41s
user 97,55s system 45% cpu 6:14,02 total

Note that I have a poor quality line with 35Kb max. So I believe when
pipelining is enabled results are limited by connection speed.

The only thing I really do not like in libwww is that it is not
maintained upsream. And it looks like it will be removed from Debian,
see bug #440436. It is suggested to move to libcurl. So I did some
more experiments ant came to conclusion that doing the same with
libcurl is possible but would require much more work. I think adding
proper pipelining support to haskell http library is the way to go. I
plan to do some research on this topic.

Regards,
  Dmitry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: darcs_libwww.diff
Type: text/x-patch
Size: 6384 bytes
Desc: not available
Url : http://lists.osuosl.org/pipermail/darcs-devel/attachments/20071221/470914bf/attachment.bin 


More information about the darcs-devel mailing list