[darcs-users] darcs patch: switch Darcs.Patch.FileName to be ByteString.Char8 int...
Simon Marlow
marlowsd at gmail.com
Fri Sep 25 13:58:57 UTC 2009
On 25/09/2009 14:25, Jason Dagit wrote:
>
>
> On Fri, Sep 25, 2009 at 4:34 AM, Simon Marlow <marlowsd at gmail.com
> <mailto:marlowsd at gmail.com>> wrote:
>
> On 24/09/2009 04:57, Jason Dagit wrote:
>
> I ran this more with normal builds and just the GC statistics.
> Here is
> what we see:
> unmodified darcs:
> 469 MB total memory in use (3 MB lost due to fragmentation)
> Total time 26.51s ( 29.76s elapsed)
> Productivity 85.6% of total user, 76.2% of total elapsed
>
> 469 MB total memory in use (3 MB lost due to fragmentation)
> Total time 26.59s ( 32.47s elapsed)
> Productivity 85.6% of total user, 70.1% of total elapsed
>
> 469 MB total memory in use (3 MB lost due to fragmentation)
> Total time 26.54s ( 29.95s elapsed)
> Productivity 85.5% of total user, 75.8% of total elapsed
>
> With my patch applied:
> 554 MB total memory in use (4 MB lost due to fragmentation)
> Total time 23.30s ( 31.56s elapsed)
> Productivity 83.1% of total user, 61.3% of total elapsed
>
> 554 MB total memory in use (4 MB lost due to fragmentation)
> Total time 22.85s ( 26.33s elapsed)
> Productivity 82.9% of total user, 71.9% of total elapsed
>
> 554 MB total memory in use (4 MB lost due to fragmentation)
> Total time 22.88s ( 26.38s elapsed)
> Productivity 82.8% of total user, 71.8% of total elapsed
>
> Now that the profiler is disabled the productivity with my
> changes is
> less, the run-time is maybe improved by 2-3 seconds, and the memory
> usage has increased by almost 100 megs. I can only assume that the
> profiling is interfering with my results a fair bit.
>
>
> The profile graphs in your previous message showed a residency of
> around ~30M before your patch, and ~6M after your patch. Which
> seems like a worthwhile saving. I presume those graphs were from a
> different test case? If not, then something very strange is going
> on. If they are from a different test case, then do the numbers
> stand up when using the non-profiled darcs?
>
>
> Thanks for trying to help me understand the profiling.
>
> My test case has not changed at all. The only thing I've been changing
> is the darcs source code.
>
> Do you have suggestions on determining which strange thing may be happening?
Well, if the heap profile is showing a ~30M residency and yet the
runtime is using 500MB, there's about 400MB unaccounted for.
Ah - so one problem is that the contents of ByteStrings are not
accounted for in the heap profile, for unpleasant technical reasons to
do with the way they're stored. Unfortunately it's not easy to fix,
because the GC doesn't retain information about which ByteStrings are
live, only that the 4KB block in which they live is still alive.
The underlying problem in your example may be fragmentation, due to the
way that ByteStrings are pinned and hence hold on to the whole 4KB block
in which they were allocated until they die. Duncan has been thinking
about how to improve the situation, but I'm not sure of the current
status - Duncan?
Cheers,
Simon
More information about the darcs-users
mailing list