[darcs-devel] [issue80] memory usage is 3X patch size, and darcs record took at least 35 minutes

Zooko bugs at darcs.net
Wed Jan 4 09:14:56 PST 2006


New submission from Zooko <zooko at zooko.com>:

I created a 0.5 GiB file and darcs added it, darcs consumes 1.5 GiB of RAM
during the darcs add and again during the darcs record.  It also took 35
minutes of maximum CPU on my high-powered workstation before I killed it.

It would be nice if darcs required enough RAM to store "only" one copy of the
patch.  It would be nicer if darcs required less RAM -- using instead a fixed
maximum buffer of RAM and lazily processing the file as needed.

Hopefully the fact that laziness is one of the oldest, core features of the
design of Haskell means that it is relatively easy for programmers to implement
algorithms that do not eagerly consume RAM ?

The extreme CPU usage is perplexing.  Are we trying to match the entire
contents of the binary file against a regex or something?

Regards,

Zooko

DARC yumyum:/mnt/sdb1/zooko/tmp$ time head --bytes=`python -c 'print 2**29'` /dev/zero > 0.5_GiB_file_a

real    0m1.790s
user    0m0.116s
sys     0m0.820s
DARC yumyum:/mnt/sdb1/zooko/tmp$
DARC yumyum:/mnt/sdb1/zooko/tmp$ l
drwxr-xr-x  5 zooko zooko       200 Jan  4 12:29 ./..
drwxrwxr-x  6 zooko zooko       184 Jan  4 12:36 ./_darcs
drwxrwxr-x  3 zooko zooko       104 Jan  4 12:36 ./.
-rw-rw-r--  1 zooko zooko 536870912 Jan  4 12:36 ./0.5_GiB_file_a
DARC yumyum:/mnt/sdb1/zooko/tmp$ darcs add 0.5_GiB_file_a
DARC yumyum:/mnt/sdb1/zooko/tmp$ time darcs record
addfile ./0.5_GiB_file_a
Shall I record this patch? (1/?) [ynWsfqadjkc], or ? for help: y
binary ./0.5_GiB_file_a
Shall I record this patch? (2/?) [ynWsfqadjkc], or ? for help: y
What is the patch name? a
Do you want to add a long comment? [yn] n

Couldn't handle interrupt since darcs was in a sensitive job.
Couldn't handle interrupt since darcs was in a sensitive job.
Finished recording patch 'a'

real    35m49.529s
user    34m33.986s
sys     0m19.432s
DARC yumyum:/mnt/sdb1/zooko/tmp$

----------
messages: 300
nosy: droundy, tommy, zooko
status: unread
title: memory usage is 3X patch size, and darcs record took at least 35 minutes

____________________________________
Darcs issue tracker <bugs at darcs.net>
<http://bugs.darcs.net/issue80>
____________________________________




More information about the darcs-devel mailing list