[darcs-devel] Database-based current [was: darcs patch]

[David Roundy]

On Fri, Sep 17, 2004 at 07:04:51PM +0200, Juliusz Chroboczek wrote:
> > This is definitely a contribution that I'm excited about, although I
> > haven't had time to look at it in detail.
> 
> Thanks for the kind words.  I'm looking forwards to getting a list of
> bugs from you ;-)

:) Not yet, though, I've been getting behind on my darcs email and have
catching up to do (plus in an hour I'm going to go shovel some dirt...).

> > But it looks like you've done about half (or a third?) of the work
> > to implement a database-stored Current.
> 
> If by database you mean Berkeley DB, then I'd like to argue against
> it.  I think a simple archive format, such as tar or zip, would be a
> much better idea.  (If anyone knows of a library that writes zipfiles,
> has a license that is compatible with Darcs and is portable to
> Windows, I'll be glad to implement such a beast.)

I don't mean specifically Berkeley DB, but I think a custom format with
some sort of DB library would probably serve better than a format like tar
or zip (although I like the idea of supporting multiple database formats,
and tar would be very appealing).  In particular, slurp gets the list of
all the files and their sizes and modification dates strictly (i.e. not
lazily), so storing that information all together would improve speed,
while reading the listing of contents of a tar involves reading the entire
file (albeit perhaps skipping sections).  I don't know about zip, but I
don't know if zip would let you store the modification times, which is a
key optimization.

In any case, once we have the framework for one database, supporting
another shouldn't be hard.

> > the CVS repository had a file called System.cpp and another called
> > system.cpp at the same time.
> 
> I'm not sure the current implementation would solve this problem; it
> still assumes you can construct things in /tmp.  I think you'd need to
> implement a version of
> 
>   withTempFile foo (apply_patches >> slurp)
> 
> that works entirely in core (see for example the hunk in my patch in
> Population.lhs).

Yeah, perhaps I overstated things a bit.  But still, I'm sure your work
will make it a *bit* easier to put in the hooks for a database current.

You'd still need to rewrite the SlurpDirectory functions to optionally
operate on a database.  For reading, this will be pretty easy, since
reading is always done via the same function, but writing will be more of a
pain, since there are write_slurp, write_slurp_dirty, and
write_and_read_slurp_dirty.

On the plus side, once we have that generalization, it can be applied to
more than just current, it can also be applied to functions like check,
which need to write a whole bunch of intermediate files, and which can be
very time-consuming.
-- 
David Roundy
http://www.abridgegame.org