[darcs-users] darcs patch: add exception to haskell_policy.sh for D... (and 61 more)

David Roundy droundy at darcs.net
Thu Oct 30 20:00:32 UTC 2008


On Wed, Oct 29, 2008 at 07:50:32PM +0000, Eric Kow wrote:
> On Wed, Oct 29, 2008 at 10:30:12 -0700, Jason Dagit wrote:
> > I want to see benchmarks too, but I thought I would justify why we
> > expect this to be no slower than the previous code...Everything below
> > is stuff that we discussed during the Sprint.
> 
> Well, attached is a second set of comparative timing tests, sorry, only
> run once and with no nice output yet.  Hopefully you can use a graphical
> diff tool to do side by side comparison.
> 
> A nice little summariser script, maybe using the Haskell tabular
> library might be handy

I've run my own set of timings, which give considerably more dramatic
differences than yours show, perhaps because I ran my tests on large
repositories?

I'll summarize my results up here, but you can look below for a more
verbose summary.  The new code is almost always faster by my reckoning, and
often *much* faster (as much as a factor of three!).

I do, however, observe a performance regression on darcs annotate Setup.hs
(run in the darcs repository).  It's a small regression (smaller than
Eric's tests show), but reproducible.  And it's all the more striking given
the dramatic improvement the new code shows in all the other tests, which
suggest there may be a single function that has a large performance
regression (since it seems likely that parts of the annotate command have
been sped up in the new version).  I don't have the time or inclination to
track down this issue, but I do hope Don is interested enough to look into
it!

I expect I'll apply this bundle once I've looked over the actual code...

David

Details of the timings I did...
===============================

I tested with three binaries:

fps:  darcs before salvo 8, no bytestring
old:  darcs before salvo 8, with bytestring
new:  darcs after salvo 8 (with bytestring, by definition)

These tests were done with the ghc 6.8.2 packages that ship with Ubuntu
Hardy Heron on an x86_64 processor, or with the ghc 6.6 in Debian Etch on a
Pentium III.  I'm giving the faster wallclock time out of two tries, in
seconds.  Runs of the same test for various darcsen were interleaved to
hopefully minimize caching effects between runs.  Note that the ghc 6.6
tests don't include the "old" bytestring possibility, since I didn't
backport the fixes that allowed darcs to work with the version of
bytestring shipped with ghc 6.6.

I'll summarize here:

annotate on franchise (6.8.2)
============================

(6.8.2)
fps: 0.24
old: 0.24
new: 0.18
(6.6)
fps: 1.80
new: 1.31

annotate on darcs repo (6.8.2)
==============================

(6.8.2)
fps: 17.25
old: 22.10
new: 18.74
(6.6)
fps: 83.47
new: 76.35

check on darcs repo
===================

(6.8.2)
fps: 36.95
old: 43.77
new: 21.23
(6.6)
fps: 156.99
new: 126.85

get on DFT++ old-fashioned repository (6.8.2)
=============================================

(6.8.2)
fps: 19.22
old: 18.90
new: 7.15
(6.6)
fps: 86.38
new: 66.24


More information about the darcs-users mailing list