bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. #3372
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In Python 3, the buffer and the underlying file object are separate and so the order in which objects are finalized matters. This is unlike Python 2 where the file and buffer were a single object and finalization was done for both at the same time. In Python 3, if the file is finalized and closed before the buffer then the data in the buffer is lost. This change adds a doubly linked list of open file buffers. An atexit hook ensures they are flushed before proceeding with interpreter shutdown. This is addition does not remove the need to properly close files as there are other reasons why buffered data could get lost during finalization. Initial patch by Armin Rigo.
The previous implementation was not careful enough to avoid causing issues in multi-threaded cases. Check for buf->ok and buf->finalizing before actually doing the flush. Also, increase the refcnt to ensure the object does not disappear.
pitrou
approved these changes
Sep 6, 2017
|
Sorry for not having caught this while reviewing, but the change here doesn't work. The problem is that |
nascheme
added a commit
to nascheme/cpython
that referenced
this pull request
Dec 13, 2017
We can't use _Py_PyAtExit() as it only supports registering a single callback. It is used by the atexit module and so we can't use it. We can't use Py_AtExit() either because it calls functions too late in the interpreter shutdown process. Instead, create io._flush_all_buffers. In io.py, register it with the atexit module.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a fixed (hopefully) version of my previous patch. Checking buf->ok and buf->finalizing is necessary, as suggested by Antoine Pitrou.
https://bugs.python.org/issue17852