[darcs-users] DeleteFile on Windows (was: Re: hashed-storage work now merged in (woo!))

[Jason Dagit]

Now, I think we can introduce POSIX file semantics on win32.  Just check out
the articles I found.  Particularly the second one.

On Thu, Oct 8, 2009 at 3:47 PM, Jason Dagit <dagit at codersbase.com> wrote:

> Thanks Petr.  I'm trying to wrap my head around this so I have some
> questions.
>
> On Thu, Oct 8, 2009 at 3:17 PM, Petr Rockai <me at mornfall.net> wrote:
>
>> Hi,
>>
>> Eric Kow <kowey at darcs.net> writes:
>> > So you may have noticed me saying this in a couple of recent threads.
>> > Petr Ročkai's hashed-storage work from his 2009 Google Summer of Code
>> > project has been merged!
>> >
>> > I thought I would take a few moments to give everybody an overview of
>> > how this work benefits us, and where we'll be going in the future.
>>
>> before we can really celebrate, there're some issues that we need to sort
>> out. Unfortunately, darcs has regressed a fair bit on win32, as our
>> current red
>> Windows slave witnesses. I can reproduce some of these failures on my
>> virtualboxed Windows XP, but not even all of them. It won't be easy to
>> fix,
>> either. I suspect part of the problem is in more aggressive mmapping -- in
>> fact, we now mmap pristine on Windows, which was never before the case, as
>> far
>> as I can tell.
>>
>> The mmap package in its current incarnation uses C finalizers for mmap'd
>> areas,
>> but that doesn't seem to be enough to get the mappings finalized soon
>> enough
>> (with regards to deleting files in pristine).
>>
>
> I want to see if I understand the situation correctly.
>
> * On POSIX even if a file is open you can write over it, on windows this is
> not the case
> * mmaping a file keeps it open until the memory is unmmaped
> * finalizers are used to unmmap the memory, and hence unlock files on
> windows
> * GHC is allowed to delay finalizers all the way until the program has
> terminated and the RTS is cleaning up
>
>
>
>>
>> -- | Remove a file. On POSIX, this is alias to removeFile. On Windows, we
>> try
>> -- to removeFile first, but if that fails with permission problems
>> (probably
>> -- still open), we instead move that file into the trash location set
>> using
>> -- "setTrashLocation".
>> trashFile :: FilePath -> IO ()
>> trashFile = undefined
>>
>
> This appears to assume:
> * On windows you can move a file even if it is open
>
> Is that assumption true?
>
> What about telling GHC to run any pending finalizers before trying to
> remove files?
>
> Can we be absolutely sure that windows doesn't have some sort of extended
> file system api that allows the semantics we need?  What if this isn't the
> real problem?  How did you determine that it was the finalizers at fault?
>

I did a quick bit of googling and some things of interest:
http://www.conifersystems.com/2008/10/21/windows-vs-unix-file-system-semantics/

That article says:

> *The Windows delete and rename model is different.*  You wouldn’t know
> this from the Win32 APIs, but in order to delete or rename a file in
> Windows, you first have to open it!  Once you’ve opened it can you call
> NtSetInformationFile with InformationClass of FileDispositionInformation or
> FileRenameInformation.  Setting FileDispositionInformation doesn’t even
> delete the file; it merely enables delete-on-close for the file, and the
> delete-on-close request could very well be cancelled later.
>
> *File sharing restrictions and locking are different.*  Unix generally
> avoids the idea of restricting what can be done with a file just because
> someone else is using it.  Having a file open doesn’t prevent it from being
> unlinked, and two people can open the same file for writing.  On Windows,
> all of this is true in theory — you can request whatever sharing mode you
> want when you open a file — but in practice, most applications use
> restrictive sharing modes, preventing two apps from using the same file at
> the same time.  Inside a single file, we also have byte range locking.
> Windows uses mandatory locking: if someone else has the bytes locked, an
> attempt to modify those bytes with WriteFile() will fail (but this is not
> enforced for memory-mapped files!).  Unix uses only advisory locking and
> makes no effort to error-check read() or write() calls; it assumes that the
> application will be responsible and won’t touch data it hasn’t first locked.
>
So maybe we need to specify a different sharing mode when we open files that
will get mmapped and possibly deleted?

Also, in that article is this interesting bit:

*The Windows cache manager holds files open.*  When you close the last
> handle to a file, from the file system driver’s point of view, the file may
> still be open.  This makes it very difficult to unload a Windows file system
> driver without a reboot.  Unmounting it, i.e., removing the drive letter
> symbolic link, is easy, but until memory pressure forces the Windows cache
> manager to flush its cached mappings, those cached mappings may stay around
> indefinitely.
>

Could this cache manager folding files open be a reason you can't delete
them?  That seems rather odd to me, but I thought I'd throw it out there.

Then I found this gem:
http://mg.to/2004/09/30/file_share_delete-in-shell-extension

According to that article we just need to pass FILE_SHARE_DELETE to
CreateFile when you open the file (the win32 api has funny names for
functions, just read the article and you'll see why fopen won't work here).

This article has other more exotic ways to delete mmaped files, although in
their case they want to make self-deleting executables:
http://www.catch22.net/tuts/selfdel

So, I think we can fix this by changing/adding code in Workaround.hs.

HTH,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/darcs-users/attachments/20091008/038f66b7/attachment.htm>