[darcs-devel] [issue2607] darcs record is too slow with medium/large numbers of files

[Simon Michael]

Simon Michael <simon at joyful.com> added the comment:

Thanks for looking at it Ben. Yes, I made one as I was having trouble
reproducing consistent results, and wanted to be sure before reporting.
Here's the scripts:


# Makefile.linux

TIME=/usr/bin/time -f%E
DARCS=darcs

test:
	for N in \
	 100 200 300 400 500 600 700 800 900 \
	 1000 2000 3000 4000 5000 6000 7000 8000 9000 \
	 ; do printf "[`date +%H:%M:%S`] recording $$N files: "; printf "`make
-s test$$N`" ; done
	 # 10 20 30 40 50 60 70 80 90 \
	 # 100 200 300 400 500 600 700 800 900 \
	 # 1000 2000 3000 4000 5000 6000 7000 8000 9000 \

# time darcs operations in a darcs repo with this many files
test%:
	@make -s t$*
	- at cd t$*; \
		rm -rf _darcs; \
		$(DARCS) init -q; \
		$(DARCS) add -q *; \
		$(TIME) $(DARCS) record -am 'record' -q >/dev/null
#		( time ( $(DARCS) record -am 'record' ) ) 2>&1 | grep real

# make a test directory with N files. Must be the last make target
beginning with t.
t%: dict
	mkdir -p $@; cd $@; for w in $$(head -$* ../dict); do echo $$w > $$w; done

# make a dictionary of simple unique ascii words
dict:
	strings /usr/share/dict/words | grep -v \' | sort | uniq > $@

clean:
	rm -rf t[0-9]*/_darcs


# Makefile.mac
DARCS=darcs

test:
	for N in \
	 100 200 300 400 500 600 700 800 900 \
	 1000 2000 3000 4000 5000 6000 7000 8000 9000 \
	 ; do printf "[`date +%H:%M:%S`] recording $$N files: "; echo `make -s
test$$N` ; done
	 # 10 20 30 40 50 60 70 80 90 \
	 # 100 200 300 400 500 600 700 800 900 \
	 # 1000 2000 3000 4000 5000 6000 7000 8000 9000 \

# time darcs operations in a darcs repo with this many files
test%:
	@make -s t$*
	cd t$*; \
		rm -rf _darcs; \
		$(DARCS) init >/dev/null; \
		$(DARCS) add -q *; \
		( time ( $(DARCS) record -am 'record' ) ) 2>&1 | grep real

# make a test directory with N files. Must be the last make target
beginning with t.
t%:
	mkdir -p $@; cd $@; for w in $$(head -$* /usr/share/dict/words); do
echo $$w > $$w; done

clean:
	rm -rf t[0-9]*/_darcs


As you can see, it wasn't that easy to get clean output on two platforms. :)

__________________________________
Darcs bug tracker <bugs at darcs.net>
<http://bugs.darcs.net/issue2607>
__________________________________