[darcs-users] [PATCH] darcs-benchmark: better tracking of binaries

[Petr Rockai]

Hi,

this bundle primarily implements tracking of binaries based on their
--exact-version. This makes the "darcs-benchmark report" command *much* more
robust. If I ran darcs-benchmark against my default darcs (which is HEAD most
of the time) multiple times, with recompiling/reinstalling in between, the
output would now look like this:

[snip]

   darcs 0: 2.4.98.1 (+ 103 patches),
            2010-08-06 13:27:39, GHC 6.12.1
   darcs 1: 2.4.98.1 (+ 109 patches),
            2010-08-07 01:51:04, GHC 6.12.1
 darcs-2.3: 2.3.0 (release),
            2009-08-11 16:03:11, GHC 6.10.4

[snip]

======  =====  =======  ==========  =======  ==========  =======
          2.3     sdev  op darcs 0     sdev  op darcs 1     sdev
======  =====  =======  ==========  =======  ==========  =======
    wh  6.3ms  (0.3ms)       6.9ms  (0.8ms)       6.6ms  (0.6ms)
wh mod  9.6ms  (0.5ms)      11.2ms  (1.2ms)      11.2ms  (1.2ms)
 wh -l      -        -       7.7ms  (0.4ms)       7.7ms  (0.3ms)
======  =====  =======  ==========  =======  ==========  =======

[...]

You can see that the two builds are now correctly separated. You can also
notice that the binaries automatically get a fairly detailed description in the
output.

Apart from sheer robustness, the importance of this feature lies with
automation: we run darcs-benchmark on buildbot after each checkin. With these
patches, darcs-benchmark can start collecting data that is properly separable:
each build will get its own numbers (right now, everything is mashed into a
single lump called "darcs-HEAD"). From that data, we could start plotting
performance per buildbot run automatically (this will need a new output format
for report, presumably).

Yours,
   Petr.

PS: There's a couple of unrelated patches in here -- since one of those got
pulled in through dependencies, I figured I could as well include all of
them... The above is implemented by the last 2 patches.

5 patches for repository code.haskell.org:/srv/code/darcs/darcs-benchmark:

Wed Jul 28 11:21:13 CEST 2010  Petr Rockai <me at mornfall.net>
  * Remove extra "mod" bits from benchmark names.

Wed Jul 28 12:25:30 CEST 2010  Petr Rockai <me at mornfall.net>
  * Add a "git" repo variant, created using tailor.

Wed Jul 28 23:45:14 CEST 2010  Petr Rockai <me at mornfall.net>
  * Improve tailor conversion resilience a bit.

Sat Aug  7 01:29:55 CEST 2010  Petr Rockai <me at mornfall.net>
  * Keep track of exact versions of test binaries.

Sat Aug  7 01:54:43 CEST 2010  Petr Rockai <me at mornfall.net>
  * Disambiguate different binaries with same name in reports.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/x-darcs-patch
Size: 30970 bytes
Desc: A darcs patch for your repository!
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20100807/c5613606/attachment-0001.bin>