[darcs-users] Darcs 2.4 - performance testing (stage 0)

Eric Kow kowey at darcs.net
Thu Dec 24 17:13:11 UTC 2009


Hi all,

Thanks to all the volunteers who signed up to participate in darcs
benchmarking.

Benchmark volunteers found!
----------------------------------------------------------------------
So far we have

 Max Battcher : Windows Vista
 Nathan Gray  : Linux
 Jason Dagit  : MacOS X

We have also some backup volunteers

 Stephan Günther : MacOS X (Tiger)
 Stephan Günther : Linux
 Jason Dagit     : Windows 7

There's no real distinction between the first and backup volunteers (the
more the merrier); mostly just a matter of who I think will have an
easier time at it.

Request zero: please install the following software
----------------------------------------------------------------------
Could all volunteers please ensure that you can cabal install the
following:
   
   - darcs-2.3.1 (manually rename this afterwards to darcs-2.3.1)
   - darcs-benchmark
   - HEAD darcs

It may also help to make a test run

1. Make a benchmarking directory
2. darcs-benchmark --get
3. Run darcs-benchmark comparing darcs-2.3.1 and darcs HEAD

The results will not be very useful yet, because we're still waiting on
the amended version of Luca's http://bugs.darcs.net/patch72 to make it
in

Benchmarking goals
----------------------------------------------------------------------
I think the three questions we need to answer are, in order of
priority.

1. How do Darcs 2.3.1 and Darcs 2.4 compare on hashed repositories?
   I think we can do this with currently existing tools.

   NB. Darcs 2.4 also introduces the darcs optimize --pristine command
   which rearranges the pristine cache of hashed repositories.  Perhaps
   we should also have an extra run to compare unoptimised and optimised
   pristine too.

2. How do Darcs 2.3.1 with an old-fashioned repository compare with
   Darcs 2.4 with a hashed repository?

   This is quite important because we aim for hashed repositories
   in Darcs 2.4 to be good enough for the GHC Team to switch over
   completely, for example, replacing their checkpoint and --partial
   based buildbot workflow with get --lazy
   http://lists.osuosl.org/pipermail/darcs-users/2009-September/021370.html

   Unfortunately, it's not clear how to do this yet.  I think we need
   some more work on darcs-benchmark and soon.

3. How does Darcs 1.0.9 with an old-fashioned repository compare with
   Darcs 2.4 on a hashed repository?

   This is useful for organisations which are still deliberately using
   darcs 1.0.9.
   
   It's also good to give us an idea of how much progress we've made (or
   how much ground we're catching up on) since the days when the Darcs
   code only had old-fashioned repositories in mind.

   I suspect that this will be easier to accomplish once we have done
   the work for #2.

I think #1 and #2 are must haves, and #3 is a would-be-nice
(but perhaps an easy one if we can do #2).  (Please let me know
if I've got the wrong benchmarking questions in mind).

That's all for now!  I'll send a stage 1 mail when we have a clearer
idea how to accomplish goal #2

Thanks, everyone :-)

-- 
Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
PGP Key ID: 08AC04F9
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20091224/df3b057f/attachment.pgp>


More information about the darcs-users mailing list