[darcs-users] darcs patch: switch Darcs.Patch.FileName to be ByteString.Char8 int...

Simon Marlow marlowsd at gmail.com
Fri Sep 25 13:58:57 UTC 2009


On 25/09/2009 14:25, Jason Dagit wrote:
>
>
> On Fri, Sep 25, 2009 at 4:34 AM, Simon Marlow <marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>> wrote:
>
>     On 24/09/2009 04:57, Jason Dagit wrote:
>
>         I ran this more with normal builds and just the GC statistics.
>           Here is
>         what we see:
>         unmodified darcs:
>         469 MB total memory in use (3 MB lost due to fragmentation)
>         Total time   26.51s  ( 29.76s elapsed)
>         Productivity  85.6% of total user, 76.2% of total elapsed
>
>         469 MB total memory in use (3 MB lost due to fragmentation)
>         Total time   26.59s  ( 32.47s elapsed)
>         Productivity  85.6% of total user, 70.1% of total elapsed
>
>         469 MB total memory in use (3 MB lost due to fragmentation)
>         Total time   26.54s  ( 29.95s elapsed)
>         Productivity  85.5% of total user, 75.8% of total elapsed
>
>         With my patch applied:
>         554 MB total memory in use (4 MB lost due to fragmentation)
>         Total time   23.30s  ( 31.56s elapsed)
>         Productivity  83.1% of total user, 61.3% of total elapsed
>
>         554 MB total memory in use (4 MB lost due to fragmentation)
>         Total time   22.85s  ( 26.33s elapsed)
>         Productivity  82.9% of total user, 71.9% of total elapsed
>
>         554 MB total memory in use (4 MB lost due to fragmentation)
>         Total time   22.88s  ( 26.38s elapsed)
>         Productivity  82.8% of total user, 71.8% of total elapsed
>
>         Now that the profiler is disabled the productivity with my
>         changes is
>         less, the run-time is maybe improved by 2-3 seconds, and the memory
>         usage has increased by almost 100 megs.  I can only assume that the
>         profiling is interfering with my results a fair bit.
>
>
>     The profile graphs in your previous message showed a residency of
>     around ~30M before your patch, and ~6M after your patch.  Which
>     seems like a worthwhile saving.  I presume those graphs were from a
>     different test case?  If not, then something very strange is going
>     on.  If they are from a different test case, then do the numbers
>     stand up when using the non-profiled darcs?
>
>
> Thanks for trying to help me understand the profiling.
>
> My test case has not changed at all.  The only thing I've been changing
> is the darcs source code.
>
> Do you have suggestions on determining which strange thing may be happening?

Well, if the heap profile is showing a ~30M residency and yet the 
runtime is using 500MB, there's about 400MB unaccounted for.

Ah - so one problem is that the contents of ByteStrings are not 
accounted for in the heap profile, for unpleasant technical reasons to 
do with the way they're stored.  Unfortunately it's not easy to fix, 
because the GC doesn't retain information about which ByteStrings are 
live, only that the 4KB block in which they live is still alive.

The underlying problem in your example may be fragmentation, due to the 
way that ByteStrings are pinned and hence hold on to the whole 4KB block 
in which they were allocated until they die.  Duncan has been thinking 
about how to improve the situation, but I'm not sure of the current 
status - Duncan?

Cheers,
	Simon


More information about the darcs-users mailing list