[darcs-devel] darcs patch: add test that triggers "too many open fi... (and 2 more)

[Ian Lynagh]

On Sat, Aug 27, 2005 at 03:37:48PM -0400, David Roundy wrote:
> 
> [avoid reading patches we just wrote when pulling.
> David Roundy <droundy at darcs.net>**20050827192349] 
> <
> > {
> hunk ./Pull.lhs 253
>  save_patches _ _ (Just []) = return []
>  save_patches _ _ Nothing = return []
>  save_patches repo opts (Just (p:ps)) =
> -    do p' <- (liftM ppt2pipt) (writePatch repo opts p)
> +    do let pinf = fromJust $ patch2patchinfo p
> +       (_,ptok) <- writePatch repo opts p
>         ps' <- save_patches repo opts $ Just ps
> hunk ./Pull.lhs 256
> -       return (p':ps')
> -    where ppt2pipt :: (Patch, PatchToken) -> (PatchInfo, PatchToken)
> -          ppt2pipt (patch, pt) = (fromJust (patch2patchinfo patch), pt)
> +       return $ (pinf, ptok) : ps'
>  \end{code}

This looks like it'll mean all the files will stay in memory until their
patchinfo's get forced later. While we might be able to hack around this
with seq's, I think it's simpler to stick with the current code.

> [use unsafeInterleaveIO in Repository.writePatch to delay opening files.
> David Roundy <droundy at darcs.net>**20050827192415
>  This patch solves the "too many open files" bug when pulling many patches.
> ] 
>                  )
> hunk ./Repository.lhs 263
>  writePatch (Repo dir _ (DarcsRepository _)) opts patch =
>      withCurrentDirectory dir $
>          do fp <- DarcsRepo.write_patch opts patch
> -           patch' <- gzReadPatchFileLazily fp
> +           patch' <- unsafeInterleaveIO $
> +                withCurrentDirectory dir $ gzReadPatchFileLazily fp
>             return (patch', DarcsPatchToken fp)
>  writePatch (Repo dir _ GitRepository) _ patch =
>      withCurrentDirectory dir $
> }

This looks like the way to go to me, although I'd be more comfortable if
we made the path absolute rather than having the working directory
change later, when this is forced and we might expect to be in a
different directory. (maybe gzReadPatchFileLazily should do the
absolutification and unsafeInterleaveIO itself?)

Oh, but make test just finished and it doesn't seem to have solved the
problem:

pull_many_files..........
#     Failed test (pull_many_files.pl at line 36)
#                   '
# Fail: _darcs/patches/20050902023736-11f6a-27cdcc80a526166fe16eb66306e306a4c78b03ca.gz: openBinaryFile: resource exhausted (Too many open files)
# 
# '
#     doesn't match '(?-xism:Finished pulling)'
pull_many_files..........NOK 1# Looks like you failed 1 test of 1.           
pull_many_files..........dubious                                             
        Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
        Failed 1/1 tests, 0.00% okay
[...]
Failed Test        Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
pull_many_files.pl    1   256     1    1 100.00%  1
Failed 1/25 test scripts, 96.00% okay. 1/179 subtests failed, 99.44% okay.


Thanks
Ian