Verified on Ubuntu 12.04.2 LTS, Precise Pangolin

Once on Drone.io, once via VM ware on a mac.

Never causes problems running directly on mac.

Could you please explain what you mean when saying "The failure happens asynchronously"? The call to "dir.deleteSync(recursive: true)" is synchronous so it should also throw an exception. Do you have async calls still using the temp directory which run after the deletion? Could there be more isolates using the tmp dir at the same time?

I could be getting this wrong, but I think that the call to deleteSync is doing something async. The error does not bubble up the call stack on the deleteSync call, but does cause a crash later.

Let be double check that, though...

The deleteSync call does nothing async by itself. It calls a function in C code which does not return until it has finished. The C code enumerates all files and directories to delete and deletes them one by one. If there is an error deleting the file or directory an exception is throws.

I've updated the code to use delete() (the async version)

Still getting the crash.

After adding debug info to the code, it seems the crash happens at the next call to Directory.createTemp()

I've tried to create a minimal repo with no luck, but I'm still getting the crash on my local Ubuntu VM and on drone.io

https://drone.io/github.com/kevmoo/bot_io.dart/11

Very weird, but very reproducible. Only on Linux. Fine on mac.

Any thoughts on this? I have another solid repro on r22072

https://drone.io/github.com/kevmoo/bot_io.dart/12

Happy to help debug.

Set owner to @whesse.
Added Started label.

I have tried to reproduce with the 22072 SDK and the Apr 24 version of bot_io, but it does not reproduce. Is this on the 32-bit or 64-bit SDK?

The error looks to me as if two calls to Directory.tempDir are returning the same directory. This would cause the error you see, when a second test tries populating the directory with files after the first test has deleted it. If you can reproduce it, could you print out the directory names returned from Directory.tempDir?

If anything could produce a race condition in the VM, it would be this test, which calls tempDir simultaneously multiple times on the same Directory('') object, cached in a static variable of TempDir.

I think that all of the asynchronous tests are running simultaneously, not one after the other, so all the calls to tempDir happen at once. Once the futures return, then the tests happen one at a time, perhaps interleaving at each future call.

The error looks like it is happening when creating the file, not when deleting it, so you could check that it happens. I would put a check in EntityPopulator._populateFileWithStream, just before openWrite, that the directory of the file exists, because I think it is exactly there that the problem is. It could also be when it is opened for checking, in _getFileSha1.

Added NeedsInfo label.

Also, notice that the error, an unhandled exception, should be able to be caught by putting an error handler on the Future or Stream that is generating it. In the tests, all the futures should be chained together, so that the exception should be catchable on the final future result of the last .then, and then it can be traced back by binary search.

Hi,

I've tried out your code, and found that there is indeed a bug in our code, but it's not as expected. When a directory-listing is performed, we list all the files, even if the stream is canceled. That means that if one uses:

var isEmptyFuture = dir.list().isEmpty;

the listing will continue after the isEmpty future was completed.

That's the problem seen on line 13 of bin/packages/bot_io/src/bot_io/io_helpers.dart. Changing it to

            return EntityValidator.validateDirectoryFromMap(dir, content)
                .length.then((l) => l == 0);

solves the issue, at it wait for the stream to complete.

However, the listing should honor the stream cancel, so it's not a bug in your code(!), this is just a workaround.

Removed the owner.
Added Accepted label.
Changed the title to: "Directory.list should honor subscription cancel.".

Set owner to @skabet.
Added Started label.

FYI: this seems to be back and reproducing on Mac

Dart VM version: 0.6.3.3 r24898 (Thu Jul 11 07:47:10 2013) on "macos_x64"

Also wanted to confirm: your work-around does address the problem. Tests are passing now on my side.

I see a code review https://codereview.chromium.org/16813006
And commit r24074

...but it seems to now exist on OSX, where I was only seeing it on Linux before

Added OpSys-OSX label.

Yes, it's an issue on how the Stream/StreamController is implemented. Atm, when you cancel a StreamSubscription (e.g. isEmpty will do that once it sees the first event), it'll return immediate after the cancel call. However, in this case, stopping an async directory listing is not a sync call, leaving us no way to delay the isEmpty completion.

What I'm working on now is to first address this issue on the Stream itself, and then fix this issue.

Added this to the M6 milestone.

Removed this from the M6 milestone.
Added this to the M7 milestone.

Removed this from the M7 milestone.
Added this to the M8 milestone.

This was fixed through a series of changes, the last one being r29454.

Note that this also fixed an issues with leaking FSs/HANDLEs when the listing was canceled.

Added Fixed label.