[darcs-devel] How to simulate a buildbot run locally

[zooko]

On Mar 8, 2008, at 9:04 AM, David Roundy wrote:

> On Sat, Mar 8, 2008 at 8:43 AM, Mark Stosberg <mark at stosberg.com>  
> wrote:
>>  I think so. Here are the commands we are asking buildbot to run:
>>
>>  f1.addStep(step.Darcs, timeout=3600, repourl='http://darcs.net/ 
>> repos/unstable', mode="copy")
...
>>  f1.addStep(step.Test, timeout=3600, command=['bash','-c','make  
>> test'])
...
>>  So, I think you can decode the shell commands it runs from that.
>
> No, unfortunately buildbot does something different than this, and we
> can't really tell what it's actually doing.

It spawns a subprocess and executes something in it. It prints to a  
web page the name of the executable, the arguments, the current  
working directory, and the environment variables, and the exit code.   
Here is the output from the "make test" step mentioned above:

http://darcs.net:8008/macos-tiger/builds/26/step-test/0

We know what happens when a process is spawned and run -- the only  
remaining uncertainty is what the buildbot does with the stderr,  
stdout, and stdin pipes of the child process.  Here is the relevant  
source code:

http://buildbot.net/trac/browser/buildbot/slave/commands.py?rev=491#L311

The call to "spawnProcess" is where Buildbot hands off the job of  
spawning a subprocess to the Twisted library.  I see just before that  
it says

         if not self.keepStdinOpen:
              self.pp.closeStdin()

I see from the script quoted above in this Mark Stosberg's message  
that the name of the class created is "Test", respectively.  Let's  
look at that class:

http://buildbot.net/trac/browser/buildbot/steps/shell.py?rev=491#L280

It does not set the "keepStdinOpen" member variable, so it must still  
be set to the default setting.  A bit of Grep Oriented Programming  
tells me that the default value is false, and I think that it can be  
set by passing a keyword 'keep_stdin_open', like this:

f1.addStep(step.Test, timeout=3600, command=['bash','-c','make  
test'] , keep_stdin_open=True)

So, my current hypothesis is that closing stdin changes the behavior  
of the perl tests.  I don't think there is anything else that differs  
between the buildslave's invocation of 'make test' and a local  
invocation of 'make test' from a bash shell.

One thing that would have helped with this debugging -- whether or  
not closing stdin turns out to be an issue -- is if the buildbot page  
that describes the step (linked above) included the disposition of  
the three standard pipes as well as all of the other relevant  
information about the process's environment.  Another thing that  
would help -- whether or not closing stdin turns out to be an issue  
-- is if buildbot didn't close stdin by default.  (That way, we  
wouldn't have to wonder whether it changes the behavior, because it  
would be the same when spawned by a buildslave or when run from an  
interactive shell by a user.)


> Perhaps it's only the
> environment that differs?

We can easily tell if any environment variables differ by comparing  
the ones listed on the buildbot page that describes the step with the  
result of 'env' in the local shell.

> Maybe it changes the default shell?

According to the buildbot page linked above the process being  
executed has argv: ['bash', '-c', 'make test'].


Regards,

Zooko