[darcs-users] issue27.sh is still sporadically buggy

[zooko]

> Indeed, and they are run with 'make bugs'.   We used to run them when
> make test (or equivalently make check) was run, but people seemed to
> get  confused by  that, so  we removed them from the default check.
> It'd be helpful to be able to see which ones are still failing, so we
> could know when the bugs are fixed.  Or on which platforms a given
> bug is present.

Hm.  The way that this is done with typical test software is to mark  
individual tests, or groups of tests as "known to fail", and then the  
test software doesn't raise an alarm if a test which is so flagged  
fails, but it does raise a nice signal saying "Hey, unexpected  
success!" when it succeeds.

Such software also typically produces a nice summary showing how many  
tests of failed, passed, or unexpectedly passed on what platforms,  
and so forth.

So, this is one of the features that using test software would give  
us.  I use Python-oriented test software in my other projects, and  
while it could be pressed into service for this purpose, I suspect  
that there may be Haskell-oriented or bash-oriented test software  
that would serve better.

Regards,

Zooko
---
http://allmydata.org -- Tahoe, the Least-Authority Filesystem
http://allmydata.com -- back up all your files for $5/month