bpo-33078 - FIX queue size on pickling error

[tomMoral]

The Queue._feed does not properly handle the size of the Queue on errors. This can lead to a situation where the Queue is considered as Full when it is empty. Here is a reproducing script:

import multiprocessing as mp

q = mp.Queue(1)


class FailPickle():
    def __reduce__(self):
        raise ValueError()

q.put(FailPickle())
print("Queue is full:", q.full())
q.put(0)
print(f"Got result: {q.get()}")

This PR fixes this behavior.

https://bugs.python.org/issue33078

[pitrou]

Thanks for the fix! A few comments below.

[pitrou]

Can you add a comment explaining why you need to adjust the semaphore's value?

[pitrou]

Should you add a check for q.qsize() as well?

[pitrou]

Typo: "handling".

[bedevere-bot]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[pitrou]

Is this check correct? Depending on when the worker thread triggers, it might return 2, no?

[tomMoral]

I think it is correct because maxsize=1 so the second q.put should return only once the failure has been triggered.

[pitrou]

That's a good point, thank you.

[miss-islington]

Thanks @pitrou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

[bedevere-bot]

3.7 branch.