- fix StreamReader to handle cancellation

- add per-receive timeout to StreamReader?

- fix sock_recv() etc. to handle cancellation

Response to comment #1:

> - a convenience function to wait for a future with a timeout

Let's say wait_for() is it.

> - a convenience function to shield a future from cancellations

Just committed shield().

> - kill as_completed()
> - maybe kill wait()???

Let's keep these for now -- their API and implementation are tricky
and there are still use cases; it's not clear that an API not
constrained by PEP 3148 can do much better.

> - or make wait() "bounce" cancellation (i.e. it doesn't pass it on but it 
does raise it)?
That's still under consideration. For now, it will just return the two
sets (done, pending) as usual.

> - maybe make gather() pass cancellation on?  (optionally?)
Done.


The stuff from comment #2 is still TODO.

wait() now bounces cancel (revision c489813e3cc9).

Still left to do:

- fix StreamReader to handle cancellation
- fix sock_recv() etc. to handle cancellation

I decided not to add a per-receive timeout to StreamReader (it's complex enough 
as it is).

What is the status of this issue?

I think I kept this open to verify that all set_result() and set_exception() 
calls are guarded by a check for cancelled() (or done(), although the latter is 
more likely to hide real bugs). I just verified that this is the case in 
streams.py. I also checked all the I/O callbacks in selector_events.py, and 
found two separate issues:

- A few places use loop.call_soon(waiter.set_result, None), which may log a 
traceback (Nikolay just brought this up on the list and in 
https://codereview.appspot.com/69870048/). TBH I wonder if these really need to 
use call_soon().

- The SSL _on_handshake() call calls self._waiter.set_exception() a few times 
without checking for self._waiter.cancelled() first.

We should do a more thorough audit (maybe the places that Nikolay fixed in his 
patch are a good starting point).

For an example of such a race condition, see http://bugs.python.org/issue21447

> For an example of such a race condition, see http://bugs.python.org/issue21447

This issue has been fixed.

> - fix sock_recv() etc. to handle cancellation

In sock_recv() of selector_events.py, I see "if fut.cancelled(): return". 

On Windows, sock_recv() is implemented with recv() which reuses the common code 
to handled overlapped futures. This code handles cancellation: the _poll() 
method ignores cancelled futures ("elif not f.done: ...").

> fix StreamReader to handle cancellation

Calls to set_result() and set_exception() on the waiter are protected by "if 
not waiter.cancelled()" in StreamReader.

> A few places use loop.call_soon(waiter.set_result, None)

This issue is now fixed with the new _set_result_unless_cancelled which can be 
used with call_soon.

> The SSL _on_handshake() call calls self._waiter.set_exception() a few times 
without checking for self._waiter.cancelled() first.
> We should do a more thorough audit (maybe the places that Nikolay fixed in 
his patch are a good starting point).

I just checked set_result() and set_exception() calls, I found 3 possible 
issues:
http://bugs.python.org/issue23197

2 issues are in the SSL handshake.

I consider that the work on this issue is now done and I prefer to work on 
issues which are more specific (like #23197).