[darcs-devel] [issue80] memory usage is 3X patch size,
and darcs record took at least 35 minutes
Zooko
bugs at darcs.net
Wed Jan 4 09:14:56 PST 2006
New submission from Zooko <zooko at zooko.com>:
I created a 0.5 GiB file and darcs added it, darcs consumes 1.5 GiB of RAM
during the darcs add and again during the darcs record. It also took 35
minutes of maximum CPU on my high-powered workstation before I killed it.
It would be nice if darcs required enough RAM to store "only" one copy of the
patch. It would be nicer if darcs required less RAM -- using instead a fixed
maximum buffer of RAM and lazily processing the file as needed.
Hopefully the fact that laziness is one of the oldest, core features of the
design of Haskell means that it is relatively easy for programmers to implement
algorithms that do not eagerly consume RAM ?
The extreme CPU usage is perplexing. Are we trying to match the entire
contents of the binary file against a regex or something?
Regards,
Zooko
DARC yumyum:/mnt/sdb1/zooko/tmp$ time head --bytes=`python -c 'print 2**29'` /dev/zero > 0.5_GiB_file_a
real 0m1.790s
user 0m0.116s
sys 0m0.820s
DARC yumyum:/mnt/sdb1/zooko/tmp$
DARC yumyum:/mnt/sdb1/zooko/tmp$ l
drwxr-xr-x 5 zooko zooko 200 Jan 4 12:29 ./..
drwxrwxr-x 6 zooko zooko 184 Jan 4 12:36 ./_darcs
drwxrwxr-x 3 zooko zooko 104 Jan 4 12:36 ./.
-rw-rw-r-- 1 zooko zooko 536870912 Jan 4 12:36 ./0.5_GiB_file_a
DARC yumyum:/mnt/sdb1/zooko/tmp$ darcs add 0.5_GiB_file_a
DARC yumyum:/mnt/sdb1/zooko/tmp$ time darcs record
addfile ./0.5_GiB_file_a
Shall I record this patch? (1/?) [ynWsfqadjkc], or ? for help: y
binary ./0.5_GiB_file_a
Shall I record this patch? (2/?) [ynWsfqadjkc], or ? for help: y
What is the patch name? a
Do you want to add a long comment? [yn] n
Couldn't handle interrupt since darcs was in a sensitive job.
Couldn't handle interrupt since darcs was in a sensitive job.
Finished recording patch 'a'
real 35m49.529s
user 34m33.986s
sys 0m19.432s
DARC yumyum:/mnt/sdb1/zooko/tmp$
----------
messages: 300
nosy: droundy, tommy, zooko
status: unread
title: memory usage is 3X patch size, and darcs record took at least 35 minutes
____________________________________
Darcs issue tracker <bugs at darcs.net>
<http://bugs.darcs.net/issue80>
____________________________________
More information about the darcs-devel
mailing list