[darcs-users] the problems with autoconf

[zooko]

Folks:

In sum, autoconf seems to be a good tool for building C code to  
various flavours of Unix, but it doesn't come with good support for  
different programming languages (Haskell) or different platforms  
(Windows with or without Cygwin).  Next, it is very difficult to  
extend or debug in order to make it support these uses better.   
Finally, there don't seem to be a lot of developers interested in  
fixing and extending autoconf.  Instead, there is a lot of energy  
going into inventing completely new replacements for autoconf.  For  
example Tom Tromey, one of the original inventors of the GNU  
autotools, is working a replacement:

http://code.google.com/p/quagmire/

There also don't appear to be a lot of autotools users on Windows.  I  
suspect that people who successfully build darcs on Windows simply  
don't try to use the build tools at all, and instead follow a  
detailed recipe in which the user manually installs and configures  
components and then manually compiled darcs.  To me, this proves the  
failure of the build tools.  I want a cross-platform build tool,  
where I can issue the same instruction ("Go forth and build yourself  
now.") and it will Just Work on all platforms.

Aside from quagmire, there are many more examples of "autotools  
replacements" out there -- I am apparently not alone in my opinion  
that autoconf is unnecessarily hard to work with.  Max Battcher  
mentioned WAFS; there are a lot of others.  I haven't surveyed them.   
If I had to decide on one thing right now it would be to take Haskell  
tools like Cabal and Franchise and extend them to do everything we  
need (this is motivated by my experience in Python world, where the  
Python-specific build tools [1, 2] do an excellent job of easily  
building Python and C/C++ code on any platform, as well as doing  
package management and resolving inter-package dependencies.  I  
expect that Haskell tools could be created or extended to be  
similarly successful).  But fortunately nobody has to decide on one  
thing right now.

Okay, the specific problems that I had with autotools yesterday were:

1.  The current darcs autoconf scripts are not reliable at figuring  
out, at configure time, what is going to be available at compile  
time.  For example, ./configure will tell me that libcurl is  
available, with various features, and then at compile time I will get  
compile time errors saying that libcurl is not available.

2.  The underlying reason for this seems to be that autoconf is  
designed with C in mind, and has lots of C-specific features  
hardcoded in.  For example, I uninstalled the C compiler from my  
system and re-ran configure, and it immediately exited with 

configure: error: C compiler cannot create executables

This shows that there is a problem in the configure script -- whether  
the C compiler can create executables is irrelevant to the question  
of whether the Haskell compiler will be able to create executables at  
build time.  Indeed, this is the source of the unreliability (issue  
#1, above), the autoconf scripts sometimes conclude at configure time  
that a certain feature is available because the C compiler can use  
it, but then it turns out at build time that the Haskell compiler can't.

3.  It is really hard to debug or extend autoconf.  This is because  
its design involves layers in which the higher layers emit code  
(based on macro expansion) that gets interpreted by the lower  
layers.  This is a neat idea because it re-uses existing tools (like  
sh and GNU make), and it works fine when the code is free of bugs,  
but when there is a bug (such as there always is when writing new  
code) it is awfully hard to figure out where the problem lies.  I  
spent all day yesterday trying to either (a) persuade it that the C  
compiler is named "ghc", or (b) write specific macros like the ones  
that droundy wrote [3] to test a Haskell compiler instead of testing  
a C compiler.  I didn't succeed at either one yet.

I really can't afford to spend another day on it, so for the moment  
my work building a statically-linked darcs executable for Windows  
which includes libcurl and SSL is stalled.  I guess I could go  
through the aforementioned process of manually configuring all the  
components and manually compiling darcs on Windows, but that doesn't  
feel right -- it doesn't make it easier for other people to build  
darcs from source and to port it to different platforms.  I would  
rather spend my time contributing to a cross-platform build  
automation tool.

Regards,

Zooko

[1] http://docs.python.org/lib/module-distutils.html
[2] http://peak.telecommunity.com/DevCenter/setuptools
[3] http://allmydata.org/trac/darcs-2/browser/aclocal.m4