[darcs-users] "darcs get" crashes (usually)

[David Roundy]

On Fri, Jul 23, 2004 at 06:45:17PM +0200, Martin Schaffner wrote:
> David Roundy wrote:
> > > > > I have a darcs repo which I can't "get", neither on Windows nor on 
>  
> > > > > Mac OS X. On the mac, darcs usually does a "Bus error",
> > > > > sometimes a Seg fault, and rarely it succeeds. It is in 
> > > > > relation to garbage collection
> > > > I will also try on Mac OS X without mmap support.
> > > 
> > > I compiled darcs without mmap support, and got exactly the same result 
> > > as with mmap support.
> > 
> > Hmmmm.  This is very strange.  Without the mmap support, this means
> > there is a bug in the compiler.  In theory, the bug could be in the
> > FastPackedString module, but that module is quite simple an very well
> > tested.
> 
> I tested with the number 1'000'000 in FastPackedString.hs decreased to
> 100'000, and the behaviour was the same as with the unmodified version.
> But when I increased it to 10'000'000, the crash didn't occur anymore,
> although an inserted putStr revealed that performGC was still called many
> times.

Interesting...

> Would it be a big thing to write a dummy FastPackedString that wraps a 
> simple String? If the current FastPackedString triggers the crash, then we 
> wouldn't see the crash when the dummy FastPackedString is used, and we 
> would be certain that it's in FastPackedString (and not in the compiler).

Not a bad idea, but it would be a bit of a pain.  I'll do this when I'm a
bit more desperate.

> Would it be easy to log all calls to FastPackedString? This way I 
> might be able to create a testcase to trigger the crash.

I've just sent a patch (to the main darcs repo) to add an --enable-debug-ps
flag to configure which will do just this.  You'll get a huge amount of
messages, which log every time a PackedString is created or deleted, giving
both the pointer of the PackedString and (on creation) the routine that
created it, as well as the associated file (if it was created by a file
read or mmap).

If you can try compiling with the latest darcs configured with
--enable-debug-ps and send me the output when it crashes, that would be
great.  This would give away your filenames, but hopefully that won't be
a problem.

> > If the bug is really in darcs, it would have to be in one of the
> > ForeignPtrs (which I think are only created in FastPackedString).  
> > There's a routine to add an extra finalizer to a ForeignPtr which you 
> > could use to add a putStr whenever one is deleted.
> 
> How exactly? Sorry, I've never done anything in Haskell before.
> Anyway, this looks to me like darcs tries to access memory that was freed 
> too early. So does only knowing when something was freed help much?

Because haskell is garbage collected, memory should never be freed too
early.  Knowing when (and which) things are being freed may help with 

> > Is there any chance I either get a login on your Mac system
> 
> Sorry, I can't give you access. I can't give you my repo, either :-(

No problem.

If I start getting desperate, perhaps you could create a sanitized version
of the repo using a perl script to remove the contents of the files from
the patches?

I'm thinking something like

s/^(-.*)[^X]/$1X/g;
s/^(\+.*)[^X]/$1X/g;

which would make all your files consist of XXXXX.  If we're lucky, this
modified repo would still show the same problems.  Since it's GC-related
we'd want to keep line lengths the same.

The perl script to do the conversion would be a bit of a pain, but could
come in very handy.  Of course, if the filenames were sensitive, then we'd
have to be more clever.

> > I've only got one device on my mac,
> > which could explain why I haven't seen this.
> 
> You can also create and mount a disk image from "disk tools" in 
> "utilities". This triggers the crash as well (for my repo).
> 
> > Does the problem only show up for one particular repository? I got that
> > impression... Perhaps there's something odd about that repo, although it
> 
> Yes, up to now it only occured in one repo. The crash appeared at the 
> same time on both mac and windows.
>
> The crash is really not "reliable":

Could you try unpulling patches from your repo one at a time to see if you
can eliminate the crash? It looks like it crashes about half the time, and
it looks like your repository is somewhat small (at least in the number of
patches), so hopefully this won't be too painful.

I'm hoping that maybe the crash started when you introduced an
"interesting" patch into your repo, which might give me a hint as to how to
reproduce it.  I'm imagining something like renaming and creating (or
deleting?) a file in the same patch.  Although the crash seems to happen
during the "Copying patches" period, which is a particularly simple bit of
code!

I've tried your suggestion of creating a disk image, but can't get it to
crash.
-- 
David Roundy
http://www.abridgegame.org