[darcs-devel] announcing darcs 2.0.0pre3

[Simon Marlow]

David Roundy wrote:
> On Wed, Jan 23, 2008 at 03:26:51PM +0000, Simon Marlow wrote:
>> There are still times when I see nothing happening, for example in the 
>> unpull test on the GHC repo (see previous messages), the last progress 
>> message I get is
>>
>> Reading patches in /64playpen/simonmar/ghc-darcs2 17040
>>
>> and it sits there for 7-8 seconds before completing.  Does this maybe shed 
>> any light on why this unpull is 2 times slower than darcs1?
> 
> I've finally got an idea what's causing this.  Actually, there are three
> operations in which the hashed repositories don't scale quite as well as
> the old format--all of which are "pretty fast", so it's only visible on
> huge repositories like yours.

Ok, I've attached unpull.log and pull.log to this message, which are 
created by:

$ darcs unpull --from-tag 2007-09-25 -a --debug 2>&1 | timestamp >unpull.log
$ darcs2 pull ../ghc-darcs2-other -a --debug 2>&1 | timestamp >pull.log

respectively.  The timestamp program is a handy little tool that prepends a 
timestamp to each stdin line (also attached, for your enjoyment).  The 
times are wall-clock, not CPU-time.

Things that stand out, first from pull.log:

   3.30: Done optimizing inventory
   3.30: P: Reading patches in /64playpen/simonmar/ghc-darcs2
   3.39: Reading patches in /64playpen/simonmar/ghc-darcs2 3/762
   3.87: P: Writing inventory
   4.03: Writing inventory 1/3 :
   5.96: Writing hash file to inventories
   7.06: P: Writing inventory : 625373dfcd54b30cda45f7d5344f53b838e7b7b8

and later:

   7.33: Done finalizing changes...
  10.09: Beginning synchronizing pristine
  10.09: Reading hash file 981818f9aa9c982f57f1560e4b4ad9e1d3fb8e87 from 
pristine
.hashed/

Is there something happening between "done finalizing changes" and 
"Beginning synchronising pristine" that isn't being logged?

Incedentally, I find this line somewhat amusing:

   7.09: Writing inventory 4/3 : c2c20935976cbcef6afd7583cd6aba1b50e0092e

and why aren't there lines for 2/3 and 3/3?

In an optimised repository, do we really have to write inventories that 
haven't changed?

Now from unpull.log:

   1.23: P: Optimizing inventory
   1.80: P: Writing inventory
   1.89: Writing inventory 1/3 :
   3.87: Writing hash file to inventories
   4.98: P: Writing inventory : 625373dfcd54b30cda45f7d5344f53b838e7b7b8

later...

   7.34: Done writing pristine
   8.47: Considering whether to test...

something happened between these two?  And later:

   8.58: Reading patches in /64playpen/simonmar/ghc-darcs2 0/362
   9.05: P: Writing inventory
   9.23: Writing inventory 1/3 :
  11.10: Writing hash file to inventories
  12.16: P: Writing inventory : 625373dfcd54b30cda45f7d5344f53b838e7b7b8

We wrote the inventory twice?  later still...

  12.24: Done cleaning pristine cache
  12.44: Cleaning out any global caches...
  15.30: Beginning reading pristine
  15.30: Done finalizing changes...
  15.30: Waiting a bit for timestamps to differ...  (!!!!!!!)
  16.31: Applying patches to working directory...
  18.16: Beginning synchronizing pristine

Perhaps the "waiting a bit for timestamps to differ" could be done by 
writing a timestamp file to the repository, and making subsequent 
operations wait if the current time isn't one second greater than the 
timestamp file?

Hope this gives you enough to go on.  Of course it looks like we're into 
diminishing returns now, and there aren't any low-hanging fruit any more, 
so I completely understand if this isn't a high priority for you.

Cheers,
	Simon

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pull.log
Url: http://lists.osuosl.org/pipermail/darcs-devel/attachments/20080128/2086f58b/attachment-0003.diff 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: unpull.log
Url: http://lists.osuosl.org/pipermail/darcs-devel/attachments/20080128/2086f58b/attachment-0004.diff 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: timestamp.hs
Url: http://lists.osuosl.org/pipermail/darcs-devel/attachments/20080128/2086f58b/attachment-0005.diff