[darcs-users] darcs patch: Fix a memory leak in Darcs.Repository.Repair.

Petr Rockai me at mornfall.net
Mon Dec 22 23:20:01 UTC 2008


Hi,

some news on the "repair eats lots of memory" problem. First, the issue has
been around since 2.0.2 at least, *before* my original hashed-repair
patches. However, the hashed repair made matters even worse, especially for 2.1
-- you might remember my benchmark table:
http://lists.osuosl.org/pipermail/darcs-users/2008-December/016511.html

One part of the problem (that was due to my sloppy coding), I have already
fixed -- that'd be the filesystem side of the problem. Now, I have done some
profiling and dug through the code for a while, and narrowed the other problem
to the way our patch repair code works.

As it were, it built a FL of patches, possibly replacing some with a repaired
version. However, since it also applied those patches, their content was
evaluated and never freed (since the list was needed at the end of
repair). What I have done now is, that we only remember the patches that were
actually changed, and then re-create the list of all patches, and only then, I
zip in those changed patches.

Since the new, re-created list of patches stays unevaluated (well, the contents
part of it, anyway), we avoid the massive memory use of the original code.

The obvious problem with my current patch is the way replaceInFL works, relying
on PatchInfo equality on the repaired patches. I unfortunately haven't had time
to verify that this is always true, or whether we really want to enforce this
as a policy. Consider this patch a DRAFT until we decide upon this issue. Also,
Jason pointed out safety issues in replaceInFL and proposed use of IsEq. I
currently don't have time to grok IsEq (I need to sleep!), but Ganesh might
shed some light into that issue and with his help, I might be able to add some
safety to that code.

The patch does pass the testsuite, but I haven't actually seen anything in the
testsuite dealing with corrupt patches, or even corrupt pristine. Haven't
looked hard.

Tue Dec 23 00:05:33 CET 2008  Petr Rockai <me at mornfall.net>
  * Fix a memory leak in Darcs.Repository.Repair.
  
  The way we repaired patches forced the whole repository into memory. We now
  only store patches that we have actually changed and "zip" them back into the
  repository using replaceInFL.

Yours, 
   Petr.

PS: Kowey, I have noticed this unapplied patch in my repo:

Wed Dec 10 07:51:38 CET 2008  Petr Rockai <me at mornfall.net>
  * Remove now-unused replacePristine.

do you recall if that's left out on purpose and I should obliterate it, or did
it get lost in the cracks, or is it waiting for something?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/x-darcs-patch
Size: 82849 bytes
Desc: A darcs patch for your repository!
Url : http://lists.osuosl.org/pipermail/darcs-users/attachments/20081223/33dcdac2/attachment-0001.bin 


More information about the darcs-users mailing list