[darcs-users] darcs-benchmark 0.1.9 - benchmarking darcs-1

Nathan Gray kolibrie at graystudios.org
Wed Aug 25 20:33:40 UTC 2010


On Wed, Aug 25, 2010 at 03:19:09PM +0100, Eric Kow wrote:
> Hi Nathan and Petr:
> 
> On Fri, Aug 20, 2010 at 17:09:14 +0200, me at mornfall.net wrote:
> > So to break down, it seems to me:
> > - whatsnew and pull 1000 are faster with darcs 2
> > - record, revert, unrevert are about the same
> > - pull 100, check, repair are faster with darcs 1
>  
> > - darcs 2 full get is slower and lazy get is faster than darcs 1 get
> 
> Thanks to Nathan and Petr for their respective benchmarking and
> distillation efforts.  I can see four things to attend to from
> this thread:
> 
> 1. benchmark for darcs add (now filed)
> 2. darcs performance regression
> 3. darcs-benchmark support for darcs-1.0.9 
> 4. notion of repository variants
> 
> For #2, is the take-home message for me that hashed repositories are
> still enough of a performance regression for you to stick with darcs
> 1.0.9 and old-fashioned?  Or is the picture slightly rosier than that?
> If it's a ":-(" for this issue, then I think we should care more about
> #3 and #4

I am pleased with where things are going.  For the first time
since 1.0.9, the 'rec mod' benchmark is about 5 seconds.  In all
previous benchmarks it has been in the range of 30 to 60 seconds.
That was a definite showstopper.

The whatsnew benchmarks continue to fall below (faster) than
1.0.9.  And record, revert, and unrevert have made amazing
progress, and are either comparable to or faster than 1.0.9.  The
pull 1000 benchmark is faster than 1.0.9, but the pull 100
benchmark is slower (114s to 210s).

The benchmarks that are still up there (worrisome) are check,
repair, and non-lazy get.  However, as they are used rather
infrequently, that may not be a showstopper.  They are all about
three times slower than 1.0.9.

I am a little bit concerned about memory usage, but have not been
tracking that as actively.

I am also concerned about add, which I believe still has a
significant regression.  I would feel much more comfortable
saying, let's upgrade, if I knew that was comparable or faster.

> For #3, I've filed a couple of tickets in the experimental
> darcs-benchmark database, (c3b/159 for 'Prelude.read: no parse'; and
> c3b/139 for explicit darcs-1 support).  I realise these bug numbers for
> an obscure in-repo tracker aren't exactly useful to you.  I'm mainly
> just trying to communicate that your reporting these things has caused
> some part of the Darcs development machinery to go "ping!", hopefully in
> a useful way.

This will be especially useful for testing the add benchmark,
when it becomes available.

> For #4, Nathan, you asked about repository variants. It's useful to note
> that these variants are only created once.  Darcs recognises a
> pre-existing variant by their prefix, eg. variant.cap-op and knows not
> to re-create it.  So for what we'd want to do, darcs-benchmark would
> already see that you have an variant.cap-old directory and not bothering
> recreating it.

Ah, yes.  I see those now.  I'm glad it is a one-time thing.  I
would never had known that I needed to add the variants line to
the config file.  That was not intuitive at all.

> > Also it seems that darcs 2 is quite memory-hungry compared to darcs 1.
> > 
> > Could you maybe share the output of "darcs chan --from-tag . --count"? The
> > record times seem quite exorbitant to me... Maybe also
> > "find -maxdepth 1 -type f | wc -l" and "find -type f | wc -l"
> 
> Petr: about #2, does Nathan's reply provide all the information you
> could use for now? [I guess I'll just leave #2 in your hands, but since
> I was replying anyway...]

I would feel more comfortable about upgrading if I knew this was
taken care of, also.

-kolibrie

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20100825/07a4a0c1/attachment.pgp>


More information about the darcs-users mailing list