[darcs-users] [patch371] Implement a test-framework-based shell h... (and 3 more)

[Petr Ročkai]

New submission from Petr Ročkai <me at mornfall.net>:

Hi,

the following patches replace our current shell harness with a new one, based
on test-framework.

Pros:

- each test is run in its own private temporary directory
- tests can be run in parallel (although not through cabal test, but the runner
  understands -j N to specify number of threads) ... this gives a substantial
  speedup (5:03 with old harness to 1:27 with the new one with -j 4)
- one-stop place to run all the tests we have (QC, HUnit, Shell)
- we outsource part of the testing infrastructure to external lib (test-framework)
- better test selection options: you can ask for "issue123" and get tests that
  contain that string (you could only use full test names with the old runner)
  .. use -t with the runner or just the string to match with cabal test; there
  may be even some globbing support or something
- colors!

Cons:

- we can no longer run any tests without -ftest (and those extra dependencies)
- test-framework does not print a summary of the failed tests at the end like
  our previous runner used to (this could be possibly hacked in, or better, we
  could request the feature in test-framework)
- the output is slightly less nice, maybe another feature request for
  test-framework (align the results in a column or something)
- (...) I may be forgetting something

NB: The test runner comes with a dependency on a not-yet-uploaded hackage
package, namely "shellish". Grab from http://repos.mornfall.net/shellish
... that code originated in darcs-benchmark and when I upload it to hackage,
I'll change darcs-benchmark to use the library too. Upload ETA couple of days,
I need to document it properly first.

Yours,
   Petr.

4 patches for repository http://darcs.net:

Fri Aug 27 22:44:48 CEST 2010  Petr Rockai <me at mornfall.net>
  * Implement a test-framework-based shell harness.
  
  This merges the two test drivers we previously had, "unit" and
  Distribution.ShellHarness (used directly by Setup), into a single "darcs-test"
  binary. This patch only goes half the way, though: the new shell harness is not
  used by "cabal test" yet. That will come as a separate patch.

Sat Aug 28 00:03:53 CEST 2010  Petr Rockai <me at mornfall.net>
  * Couple of improvements in the new shell test runner.

Sat Aug 28 00:04:14 CEST 2010  Petr Rockai <me at mornfall.net>
  * Add a --network option to darcs-test to run the network shell tests.

Sat Aug 28 00:04:54 CEST 2010  Petr Rockai <me at mornfall.net>
  * Flip "cabal test" over to use darcs-test to run the shell tests.

----------
files: implement-a-test_framework_based-shell-harness_.dpatch, unnamed
messages: 12349
nosy: mornfall
status: needs-review
title: Implement a test-framework-based shell h... (and 3 more)

__________________________________
Darcs bug tracker <bugs at darcs.net>
<http://bugs.darcs.net/patch371>
__________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: implement-a-test_framework_based-shell-harness_.dpatch
Type: text/x-darcs-patch
Size: 70565 bytes
Desc: not available
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20100827/ff04c2f5/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unnamed
Type: application/octet-stream
Size: 5 bytes
Desc: not available
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20100827/ff04c2f5/attachment-0001.obj>