[darcs-users] benchmarks

[Petr Rockai]

Zooko,

"Zooko O'Whielacronx" <zooko at zooko.com> writes:
> I still think we can set up buildbots to do this. Here is a patch to
> the GNUmakefile to make a target named "benchmark" which does I think
> everything we need.
>
> Now we can configure the buildmaster to run "make benchmark" once per
> day, or once per week, or only after no new patches have arrived in
> the last 8 hours, or other policies of your choice. Buildbot will
> automatically collect all the benchmark results and make them
> browseable on the web. Let's start with that!
the catch with this is, that 5 of the buildslaves run on a single
physical piece of hardware which even though is quite beefy (4 CPUs and
16G RAM) can only sustain so much load -- especially IO. Running 5
benchmarks in parallel is not going to be much use anyway. On the other
hand, if you look at the buildbot.darcs.net waterfall, you will see that
the "6.12.1 Debian" slave has been running the benchmarks after each
push -- for a while now (many months, maybe a year now?).

It is only running for HEAD (understandably) -- what is missing from the
setup is adding comparison in time, both from the buildbot itself and
comparison against released versions on the (preferably on the same
machine). This is not completely trivial, though. Oh, and a glittery
rendering.

Yours,
   Petr.

PS: You should only feed that "copy & paste below" section into reST
tools. The rest is for your amusement if you are running the benchmark
interactively. :) Maybe we need a --quiet option.

PPS: Ok, having looked at your output, it is indeed messed up. I think
someone will need to look into that. You could install 0.1.7 in the
meantime, which I think worked.
(cabal install "darcs-benchmark == 0.1.7")