[darcs-users] darcs benchmarking stage 3

[Eric Kow]

Hi everyone,

The benchmarking effort has made a little more progress.
See below for my next batch of requests

Recent changes
==============
darcs-benchmark 0.1.6 is now available for your benchmarking pleasure.

- Tahoe-LAFS and tabular have now been added as official benchmarks

- Optimize --pristine is now taken into account
  We also now have a framework for future repository variants,
  such as the packed one I'm hoping to see.

- We now report the mean and stddev of times (instead of minimum)
  over the set of runs.

- We now report the smaller benchmarks (eg. whatsnew) in milliseconds

- HOME directory now set to playground directory (avoids any
  effect from user's global cache? note that benchmarks are local
  only, not network-based)

- New 'adaptive' benchmarking scheme to avoid ad-hoc multipliers
  in our code (thanks to Petr for the idea).  No more "wh x 50".
  Instead we run each benchmark for as many iterations
  3 =< i =< 100 to do at least 10 seconds of benchmarking.

- New --cold feature (Linux only), forces the benchmarks to be
  run under a cold cache.

- New darcs-benchmark dist command to help you make future
  benchmark directories

Request to darcs benchmarking volunteers
========================================
Please

1. Install the latest darcs-benchmark

    cabal update
    cabal install darcs-benchmark

2. Install the latest darcs beta (2.3.99.2):

     darcs get --lazy http://darcs.net/releases/branch-2.4
     # etc

3. Fetch new repositories (tabular, tahoe) and the configuration files
  
      cd your-benchmark-dir
      rm config.*
      darcs-benchmark get # this may be slow!

4. Produce some benchmarks.

   Warning!  Benchmarking may not be several times slower than before
   (at least 2x, since we're keeping track of repository variants).

   You may want to break this into stages, starting with tabular and
   tahoe-lafs to see how things go

     darcs-benchmark run darcs-2.3.1 darcs-2.3.99.2 / tabular
     darcs-benchmark run darcs-2.3.1 darcs-2.3.99.2 / tahoe-lafs
     darcs-benchmark run darcs-2.3.1 darcs-2.3.99.2 / darcs ghc-hashed

5. Paste the result into the http://wiki.darcs.net/Benchmarks
   
Many thanks!

PS. I hope my requests are not starting to become unreasonable.
    Please let me know.  I think we're nearing the end of this
    campaign, but there may be more changes ahead as we discover
    flaws in this weekend's work.  Thanks for your patience!

-- 
Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
PGP Key ID: 08AC04F9
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20100221/a9334678/attachment.pgp>