[darcs-users] "Done removing from pending:" then a VERY long wait.

James Sleeman darcs at gogo.co.nz
Mon Jul 16 14:00:20 UTC 2012


Partly this is to partly report bug, and partly to document a solution 
where darcs is getting stuck or being very very VERY slow at  "Removing 
from pending:" or "Done removing from pending:" when recording changes, 
possibly particularly when adding new files.

Short version for people from the future (http://xkcd.com/979/); if you 
find yourself in a performance blackhole when trying to record patches, 
try doing a darcs get to create a fresh copy of the repo and record your 
changes in there.

=-=-=-=-=-=-=-=-=-=

So here's the situation,  adding a bunch of stuff to an existing 
repository, in total around 70 meg, but I tried splitting it up into 
small patches (by recording some sub directories individually) to be 
kind, up to about 10 m would be the largest total patch.

These are new files the repo has never seen before.  They are mostly 
textual.

It was taking an AGE, and I mean GLACIAL.

Without --debug-verbose it gets up to "Removing from pending: X done, Y 
queued", at which point that is the last output while it grinds to a 
halt, a CPU core spinning like mad.  For minutes.. hours.. days?  Before 
finally it realises it doesn't need to heat my office with CPU power, 
just record a patch :)

Turning on --debug-verbose actually shows that it gets one step further 
to "Done removing from pending:" before the cpu grinding begins and no 
further output is shown (for a VERY long time or until I give up on 
recording that patch, CTRL-C and try recording another subset (directory)).

Now, I created a tag right before I started this epic patch recording.  
So, on a whim I did...
   darcs get --lazy -t mytag slowrepo newrepo
   cp -rP slowrepo/newstuff newrepo/newstuff
and tried recording the new stuff from in there (again in chunks).... 
kapow, done in milliseconds.

Again, on another whim I tried even simpler...
   darcs get slowrepo anothernewrepo
   cp -rP slowrepo/newstuff anotherrepo/newstuff
and tried recording from in there.... again, all over in a snap of the 
fingers, worked perfectly.

So, in short, I have one repo that when trying to record added files 
into it takes millenia, while adding those same files to a simple "darcs 
get" (without any options) copy of that repository, takes seconds, 
milliseconds even.

So there you go people from the future; if you find yourself in a 
performance quandry when trying to record patches, try doing a darcs get 
to create a fresh copy of the repo and record your changes in there.

Further Information For Darcs Developers If It Helps
-----------------------------------------------------
darcs optimize, and optimize --reorder both run, no affect, darcs check 
reported no problems.

strace for a few seconds while the old repo it is in this churning state 
is here:
     http://sirius.gogo.co.nz/d.txt
but it doesn't look very interesting to me.

Darcs version 2.8 (static build downloaded from website), same deal with 
2.7.99.1 (Ubuntu 12.04 default).

If there is anything else I can do (aside from posting the repo, which 
unfortunately is all confidential like :-/), happy to help.

---
James Sleeman




More information about the darcs-users mailing list