[darcs-devel] Getting a repository into a slurpy

[Grant Husbands]

Jason Dagit wrote:
> Just wondering, what characterizes your 'large repositories'?

I can answer this one. The main repo he's thinking of contains 2,464 
files in 148 folders, totalling 25MB. Most are text files and they 
rarely exceed 20KB. There are no files larger than 2MB, and only a few 
larger than 1MB.

There are 1,504 patches; 1,200 of those are each, gzipped, smaller than 
1KB. People normally apply diffs to recent patches and the total size of 
the files affected by all of the patches ahead of them would very rarely 
exceed 1MB. However, the time it takes to create copies of the 25MB of 
small files is appreciable.

You don't need a repo of that size to see the effect, though. Doing a 
darcs diff on a recent patch in darcs-stable takes an appreciable amount 
of time, where all that time is spent copying files.

> Another idea I had to optimize darcs was to store hunks differently so
> that parts of patch bundles could be skipped over instead of parsed.
> I partially implemented this and found that it was actually slower

For what it's worth, this wouldn't help in our case; we normally only 
diff patches near to the current state, so the time spent parsing 
patches is trivial.

I'm not really qualified to comment on any of the rest. Any pointers 
would be handy, of course.

G.