[darcs-users] Benchmark results.

[Petr Rockai]

Petr Rockai <me at mornfall.net> writes:
> === big-zoo/darcs-repo ===
>
>           || ../branch-2.2/darcs     | ../release-2.0.2/darcs    
> ==========++=========================+===========================
>  annotate ||               16.1s 64M |                  16.3s 63M
>     check ||               16.0s  8M |                  25.4s  4M
>  get full ||               11.3s  2M |                  13.1s  2M
>  get lazy ||                2.0s  0M |                   2.2s  0M
>  pull 100 ||                9.4s  7M |                  10.3s  7M
> pull 1000 ||               74.9s 18M |                  87.8s 16M
>    repair ||               16.6s 10M |                 184.7s 44M

And here come the promised results. Still the same box (16G RAM, core 2 Xeons),
this time on a local filesystem, with global cache enabled and stored on the
same filesystem as benchmark data. I have highlighted the check and repair
lines, as this is my personal focus for this release.

=== big-zoo/darcs-repo ===
          
          || darcs-2.0.2     | darcs-2.1     | darcs-2.2+
==========++=================+===============+===============
 annotate ||       11.3s 64M |     10.8s 62M |      10.2s 64M
    check ||       19.2s  5M |     18.8s  6M |       9.4s 12M <--
 get full ||        3.5s  2M |      2.2s  2M |       7.9s  3M !!
 get lazy ||        0.4s  0M |      0.3s  0M |       0.7s  0M
 pull 100 ||        3.8s  9M |      3.6s  9M |       3.3s  9M
pull 1000 ||       30.4s 14M |     29.5s 15M |      18.2s 18M
   repair ||       30.0s 65M |     18.2s 73M |       9.0s 13M <--

=== big-zoo/frugalware-hashed ===
          
          || darcs-2.0.2      | darcs-2.1      | darcs-2.2+
==========++==================+================+===============
    check ||       76.8s  22M |     73.0s  23M |      85.2s 42M <--
 get full ||       18.7s  11M |     12.7s  10M |      57.3s  9M !!
 get lazy ||        4.4s   2M |      2.7s   2M |      10.6s  0M
 pull 100 ||       24.1s  17M |     19.7s  17M |      19.4s 20M
pull 1000 ||       45.3s  23M |     40.6s  22M |      38.9s 28M
   repair ||      154.0s 299M |   3482.6s 409M |      84.6s 42M <--

=== big-zoo/ghc-hashed ===
          
          || darcs-2.0.2      | darcs-2.1      | darcs-2.2+
==========++==================+================+===============
    check ||      109.5s  47M |    105.7s  47M |     107.5s 67M <--
 get full ||       16.2s  19M |     12.6s  18M |      41.4s 19M !!
 get lazy ||        1.3s   0M |      1.1s   0M |       3.0s  0M
 pull 100 ||       77.5s  37M |     78.3s  37M |      78.7s 36M
pull 1000 ||      152.4s  47M |    150.9s  48M |     120.6s 53M
   repair ||      616.1s 505M |    259.1s 587M |     107.6s 67M <--

=== big-zoo/small-repo ===
          
          || darcs-2.0.2    | darcs-2.1    | darcs-2.2+
==========++================+==============+==============
 annotate ||        0.0s 0M |      0.0s 0M |       0.0s 0M
    check ||        0.1s 0M |      0.1s 0M |       0.4s 9M <--
 get full ||        0.0s 0M |      0.0s 0M |       0.1s 0M
 get lazy ||        0.0s 0M |      0.0s 0M |       0.0s 0M
 pull 100 ||        0.2s 0M |      0.2s 0M |       0.2s 0M
pull 1000 ||        0.2s 0M |      0.2s 0M |       0.2s 0M
   repair ||        0.1s 0M |      0.1s 0M |       0.2s 9M <--

I am happy to note that both time and memory use for repair has went down
significantly since darcs 2.0.2 with the proposed repair/check patch. If the
patch gets in (ie. it is deemed correct or otherwise fixed), I will
unilaterally declare the following goal satisfied:

1) Work on improving the situation with repair/check performance regression
that was introduced (by me) somewhere around 2.0.2, without compromising the
improved robustness...

Also, the pull times in darcs-2.2 have improved compared to 2.0.2 and 2.1,
across the board. What however bothers me is the get time for the big
repositories (marked with exclamation marks): something's definitely wrong
there. This needs investigating before the 2.2 release for sure. I suspect a
quirk with cache handling somewhere? For now, I'm opening an issue with
bugs.darcs.net: http://bugs.darcs.net/issue1282

Thorkil, it might make sense to clear up the topics and standardise on a single
way to make a list of "release critical" issues.

Looking at "edit topics" I see:
ReleaseTarget-2009-01
OldReleaseCritical
ReleaseGoal

Yours,
   Petr.

-- 
Peter Rockai | me()mornfall!net | prockai()redhat!com
 http://blog.mornfall.net | http://web.mornfall.net

"In My Egotistical Opinion, most people's C programs should be
 indented six feet downward and covered with dirt."
     -- Blair P. Houghton on the subject of C program indentation