[darcs-devel] my problems with tests that run ssh/curl

Ben Franksen ben.franksen at online.de
Thu Jul 16 09:01:18 UTC 2020


Am 16.07.20 um 00:32 schrieb Ganesh Sittampalam:
> On 12/07/2020 09:45, Ben Franksen wrote:
>> Am 12.07.20 um 03:11 schrieb Ganesh Sittampalam:
> 
>>> It turns out that quite a few of our non-network tests nevertheless do
>>> run commands like ssh and curl for various reasons. One example is
>>> issue2545_command-execution-via-ssh-uri which is fairly obvious but
>>> there are several others.
>>>
>>> I was vaguely wondering about just making those tests into "network"
>>> tests that I then wouldn't run by default. It's not a great solution as
>>> the tests don't generally genuinely rely on the network.
>>
>> By all means, go ahead and move them to the network folder. Even if they
>> don't rely on the network proper, they still use network-related
>> functionality. (I usually run all tests anyway so it doesn't matter much
>> to me whether they are network tests or not.)
>>
>> While we're at it, we could also move the three network related shell
>> functions (serve_http, finish_http, and check_remote_http) into a
>> separate tests/network/httplib, to discourage us from using them in
>> normal test scripts.
> 
> I've sent in a patch for both of these things (patch2051). I thought
> more tests were affected but right now it looks like there's only one of
> them. However this piece of spyware behaves a bit confusingly so it's
> possible others will start breaking again in the future and I'll need to
> move some more tests over to network.
> 
> Also, either I had misremembered how slow the tests are on Windows, or
> they have got a lot slower at some point at least for me. It now takes
> 200 minutes to run all three variants single threaded :-(
> 
> I think I first mentioned this in the context of the possible turtle
> changeover, but it looks like it's independent of that. Even if I run
> them with -j 4 and run just one variant, it takes 37 mins. In contrast a
> Linux VM running in VirtualBox on the same machine takes just 17 mins
> with -j 4 to run all three variants. I may need to give up doing my
> primary development on Windows and just use it for occasional test runs.

I just checked on my laptop: with -j4 it takes 3 and a half minutes to
run the full suite. If the cpu load can be trusted that would be 7
minutes with -j1.

Don't you think we could at least try and debug this to find out *what*
makes it so slow in Windows? It could be anything, really, but my gut
feeling tells me it has to do with executing the 'bash test' command.

I have attached a patch that prints a timestamp before the exec call and
inside tests/lib (and then fails). Be aware that printing the first
timestamp badly messes up the output unless you run only a single test.
For me the time difference is minimal:

2020-07-16 08:58:40.053041515 UTC # before executing bash test
EXAMPLE (Darcs3) (PatienceDiff): [Failed]
| 2020-07-16 08:58:40.056674214 UTC # from inside tests/lib

Cheers
Ben
-------------- next part --------------
1 patch for repository http://darcs.net/screened:

patch 75e2dc421494c6bcfa52e8a026dc83c444589d56
Author: Ben Franksen <ben.franksen at online.de>
Date:   Thu Jul 16 10:53:46 CEST 2020
  * timing test
  
  Note this works only when running a single test script e.g.
  
    darcs-test -t EXAMPLE -1=no -2=no
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


New patches:

[timing test
Ben Franksen <ben.franksen at online.de>**20200716085346
 Ignore-this: 7d0a6f0b5f7740d80a5bcacfbba98cbccbf8c7ca7f54d8c51465fb9d8e3200209f1a29b032727807
 
 Note this works only when running a single test script e.g.
 
   darcs-test -t EXAMPLE -1=no -2=no
] hunk ./darcs.cabal 534
+                    time,
hunk ./harness/test.hs 18
+import Data.Time.Clock ( getCurrentTime )
hunk ./harness/test.hs 118
+     timestamp <- Shelly.liftIO getCurrentTime
+     echo_err $ pack $ show timestamp
hunk ./tests/lib 3
+date -u +'%F %T.%N %Z' >&2
+exit 1
+

Context:

[darcs.cabal: remove redundant version constraints
Ben Franksen <ben.franksen at online.de>**20200427140342
 Ignore-this: 13501612338c6df2258acfaf2f9d71c5840e14f25849f66cc874e1fd20b18e6e4c34d34789e51e31
] 
[resolve conflicts after 347aeb4b5c1eccfe00956ac318a2123bca9ef9ca
Ben Franksen <ben.franksen at online.de>**20200427135437
 Ignore-this: 9217132f46b0b7fb62c71b6202f744dcc480fbc1698f602ae2537b0566051c1b676f7790d32e5fec
] 
[resolve conflicts after 5d63e665634c964d7b23dcf9c28efc72d6a6f947
Ben Franksen <ben.franksen at online.de>**20200427125216
 Ignore-this: 47121a51606f226d64d22f3a2cde9a1661672e91d4eb7c1e0e0e292c76a9883ea42377a4ea1ef2a4
] 
[loosen upper bound for shelly to <1.10
Ben Franksen <ben.franksen at online.de>**20200421081429
 Ignore-this: f2a47bfd51083a65d445bf8c0e6d85a497d3462270d55ff1d6c4c54ce31d4cb72e281bddf733739e
 
 This is required to find a build plan with ghc >= 8.8.
 
 A comment in the cabal file claims we cannot use shelly-1.9 because of two
 open issues. However, these issues only concern windows; but for windows we
 use an even stricter bound in a separate build-depend stanza, where we
 require < 1.7.2. I have moved the comment to this other stanza. The upshot
 is that building against ghc versions >= 8.8 is currently not supported on
 windows.
] 
[document why we can't bump the shelly dependency
Ganesh Sittampalam <ganesh at earth.li>**20190920184131
 Ignore-this: 3cd9ac649ec4acc2dc92027e47d10c60
] 
[clean up Darcs.Test.Patch.Check
Ben Franksen <ben.franksen at online.de>**20200426182607
 Ignore-this: fb38c25c164f284cc6b9ab7db57a5af6a0c848cc89ab6fe1add562917512c3b132c269931b0cb388
 
 There was a comment in the code about the strangeness of how in this test
 module returning a Boolean was used to communicate a failure, and how this
 should be replaced with a more conventional way of error handling. This is
 now done, using the MaybeT monad transformer.
] 
[TAG 2.15.2
Ganesh Sittampalam <ganesh at earth.li>**20190916154842
 Ignore-this: 49a3b59b9fd79ac55ad4e54388f88b77
] 
Patch bundle hash:
9f264f4330e08c7dfce1c9a6e3c1d32272f75b9a
-----BEGIN PGP SIGNATURE-----

iHUEAREIAB0WIQS1sLTEOCbYp4iyltnTbkUxbljMlwUCXxAWMwAKCRDTbkUxbljM
l8b2AP49mNq6TSqkLHMWIreo3uC4SPR9vB+hi7wI+YVgD08QlAEAoMqSEyZNaT+b
JJrUvxwNEV3LwqGu3s/tCI3L4OqFO4U=
=rzvP
-----END PGP SIGNATURE-----


More information about the darcs-devel mailing list