[darcs-users] Darcs 2.4 - performance testing (stage 1)

[Petr Rockai]

Hi,

see also issue1731... this here is a set of two benchmarks using latest
darcs-HEAD with hashed-storage HEAD (which ought to have issue1731 fully
fixed, at least on Linux). It also demonstrates the effect of optimize
--pristine (the second table, to the right).

As can be seen, optimize --pristine can have catastrophic effect on
performance of 2.3 and older darcsen (this is, coincidentally, the same
as out-of-sync mtimes, so these times are the ones that people could
actually run into quite easily in practice with older versions).

ghc-hashed
----------
                                                            (after optimize --pristine)
+--------------------++------------------+---------------+ +------------------+---------------+
|                    || darcs-2.3        |  darcs        | | darcs-2.3        |  darcs        |
+====================++==================+===============+ +==================+===============+
|         get (full) ||     11.2s  70.0M |  12.5s  71.0M | |     10.6s  68.0M |  11.9s  70.0M |
|    get (lazy, x10) ||     13.3s   9.0M |   8.7s   8.0M | |      9.9s   8.0M |   8.7s   8.0M |
|           pull 100 ||      7.2s  35.0M |   6.2s  65.0M | |      8.4s  35.0M |   6.7s  65.0M |
|           annotate ||         -      - |      -      - | |         -      - |      -      - |
|             wh x50 ||      0.8s   0.0M |   2.4s   0.0M | |      1.3s   0.0M |   1.6s   0.0M |
|         wh mod x50 ||      3.8s   3.0M |   5.3s   3.0M | |      4.3s   3.0M |   4.5s   3.0M |
|          wh -l x20 ||      4.8s   0.0M |   2.0s   0.0M | |     57.7s   0.0M |   1.7s   0.0M |
|     record mod x10 ||     18.2s  70.0M |  17.8s  75.0M | |     28.3s  80.0M |  17.8s  74.0M |
|     revert mod x50 ||     16.9s  12.0M |  10.9s   9.0M | |    105.4s  19.0M |   9.5s   8.0M |
| (un)revert mod x10 ||     19.1s  54.0M |  16.3s  48.0M | |     57.4s  62.0M |  15.6s  48.0M |
|              check ||    123.0s 152.0M | 123.6s 177.0M | |    121.8s 142.0M | 121.6s 175.0M |
|             repair ||    122.4s 142.0M | 122.8s 173.0M | |    122.7s 152.0M | 123.3s 179.0M |
|          pull 1000 ||     26.8s 203.0M |  25.6s 197.0M | |     27.9s 188.0M |  26.3s 197.0M |
+--------------------++------------------+---------------+ +------------------+---------------+

It is mostly clear that everything apart from whatsnew has become
faster. The problem here is, that even though we have lost some ground
on whatsnew on small repositories (small in the working copy size), we
ought to have captured some in the big repository territory. Our old
project repository of about 2300 working files (i.e. about twice as big
as GHC in this measure) gives the following results for whatsnew, for a
single run:

unopt 2.3: 0.08
unopt HEAD: 0.11
opt 2.3: 0.062
opt HEAD: 0.026

Here, unopt means before optimize --pristine and opt means after.

Which means that between unoptimised 2.3 to optimised HEAD we gained
about two-fold speedup (in this particular case, anyway).

Your humble & obedient servant,
    Petr.