From 6663fb1a3876bd26e37fd180cd8ec507d6796722 Mon Sep 17 00:00:00 2001 From: Mariatta Date: Sat, 26 Nov 2016 11:09:23 -0800 Subject: [PATCH 1/3] Prep the file for conversion. Add the link to bug tracker into references section --- pep-3145.txt | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/pep-3145.txt b/pep-3145.txt index df2028fafba..8a19371cb5f 100644 --- a/pep-3145.txt +++ b/pep-3145.txt @@ -27,8 +27,8 @@ PEP Deferral: PEP Withdrawal: - This can be dealt with in the bug tracker. - A specific proposal is attached to http://bugs.python.org/issue18823 + This can be dealt with in the bug tracker. A specific proposal is + attached to [11] Motivation: @@ -42,7 +42,7 @@ Motivation: and documented [4] [5]. While communicate can be used to alleviate some of the buffering issues, it will still cause the parent process to block while attempting to read data when none is available to be read from the child - process. + process. Rationale: @@ -62,12 +62,12 @@ Reference Implementation: I have been maintaining a Google Code repository that contains all of my changes including tests and documentation [9] as well as blog detailing - the problems I have come across in the development process [10]. + the problems I have come across in the development process [10]. I have been working on implementing non-blocking asynchronous I/O in the subprocess.Popen module as well as a wrapper class for subprocess.Popen that makes it so that an executed process can take the place of a file by - duplicating all of the methods and attributes that file objects have. + duplicating all of the methods and attributes that file objects have. There are two base functions that have been added to the subprocess.Popen class: Popen.send and Popen._recv, each with two separate implementations, @@ -77,25 +77,25 @@ Reference Implementation: the Python interface for file control serves the same purpose. The different implementations of Popen.send and Popen._recv have identical arguments to make code that uses these functions work across multiple - platforms. + platforms. When calling the Popen._recv function, it requires the pipe name be passed as an argument so there exists the Popen.recv function that passes selects stdout as the pipe for Popen._recv by default. Popen.recv_err selects stderr as the pipe by default. Popen.recv and Popen.recv_err are much easier to read and understand than Popen._recv('stdout' ...) and - Popen._recv('stderr' ...) respectively. + Popen._recv('stderr' ...) respectively. Since the Popen._recv function does not wait on data to be produced before returning a value, it may return empty bytes. Popen.asyncread handles this issue by returning all data read over a given time - interval. + interval. The ProcessIOWrapper class uses the asyncread and asyncwrite functions to allow a process to act like a file so that there are no blocking issues that can arise from using the stdout and stdin file objects produced from a subprocess.Popen call. - + References: @@ -106,7 +106,7 @@ References: [2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess http://ivory.idyll.org/blog/feb-07/problems-with-subprocess - [3] How can I run an external command asynchronously from Python? - Stack + [3] How can I run an external command asynchronously from Python? - Stack Overflow http://stackoverflow.com/questions/636561/how-can-i-run-an-external- command-asynchronously-from-python @@ -133,6 +133,9 @@ References: [10] Python Subprocess Dev http://subdev.blogspot.com/ + [11] https://bugs.python.org/issue18823 -- Idle: use pipes instead of + sockets to talk with user subprocess + Copyright: This P.E.P. is licensed under the Open Publication License; From 09dd7497f44cc312281ab23ed90dda12b8e2cd7d Mon Sep 17 00:00:00 2001 From: Mariatta Date: Sat, 26 Nov 2016 11:11:24 -0800 Subject: [PATCH 2/3] Output from restify script --- pep-3145.txt | 211 +++++++++++++++++++++++++++------------------------ 1 file changed, 111 insertions(+), 100 deletions(-) diff --git a/pep-3145.txt b/pep-3145.txt index 8a19371cb5f..f5d58c4da63 100644 --- a/pep-3145.txt +++ b/pep-3145.txt @@ -5,139 +5,150 @@ Last-Modified: $Date$ Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson Status: Withdrawn Type: Standards Track -Content-Type: text/plain +Content-Type: text/x-rst Created: 04-Aug-2009 Python-Version: 3.2 Post-History: -Abstract: - In its present form, the subprocess.Popen implementation is prone to - dead-locking and blocking of the parent Python script while waiting on data - from the child process. This PEP proposes to make - subprocess.Popen more asynchronous to help alleviate these - problems. +Abstract +======== +In its present form, the subprocess.Popen implementation is prone to +dead-locking and blocking of the parent Python script while waiting on data +from the child process. This PEP proposes to make +subprocess.Popen more asynchronous to help alleviate these +problems. -PEP Deferral: - Further exploration of the concepts covered in this PEP has been deferred - at least until after PEP 3156 has been resolved. +PEP Deferral +============ +Further exploration of the concepts covered in this PEP has been deferred +at least until after PEP 3156 has been resolved. -PEP Withdrawal: - This can be dealt with in the bug tracker. A specific proposal is - attached to [11] +PEP Withdrawal +============== +This can be dealt with in the bug tracker. A specific proposal is +attached to [11]_ -Motivation: - A search for "python asynchronous subprocess" will turn up numerous - accounts of people wanting to execute a child process and communicate with - it from time to time reading only the data that is available instead of - blocking to wait for the program to produce data [1] [2] [3]. The current - behavior of the subprocess module is that when a user sends or receives - data via the stdin, stderr and stdout file objects, dead locks are common - and documented [4] [5]. While communicate can be used to alleviate some of - the buffering issues, it will still cause the parent process to block while - attempting to read data when none is available to be read from the child - process. +Motivation +========== -Rationale: +A search for "python asynchronous subprocess" will turn up numerous +accounts of people wanting to execute a child process and communicate with +it from time to time reading only the data that is available instead of +blocking to wait for the program to produce data [1]_ [2]_ [3]_. The current +behavior of the subprocess module is that when a user sends or receives +data via the stdin, stderr and stdout file objects, dead locks are common +and documented [4]_ [5]_. While communicate can be used to alleviate some of +the buffering issues, it will still cause the parent process to block while +attempting to read data when none is available to be read from the child +process. - There is a documented need for asynchronous, non-blocking functionality in - subprocess.Popen [6] [7] [2] [3]. Inclusion of the code would improve the - utility of the Python standard library that can be used on Unix based and - Windows builds of Python. Practically every I/O object in Python has a - file-like wrapper of some sort. Sockets already act as such and for - strings there is StringIO. Popen can be made to act like a file by simply - using the methods attached to the subprocess.Popen.stderr, stdout and - stdin file-like objects. But when using the read and write methods of - those options, you do not have the benefit of asynchronous I/O. In the - proposed solution the wrapper wraps the asynchronous methods to mimic a - file object. -Reference Implementation: +Rationale +========= - I have been maintaining a Google Code repository that contains all of my - changes including tests and documentation [9] as well as blog detailing - the problems I have come across in the development process [10]. +There is a documented need for asynchronous, non-blocking functionality in +subprocess.Popen [6]_ [7]_ [2]_ [3]_. Inclusion of the code would improve the +utility of the Python standard library that can be used on Unix based and +Windows builds of Python. Practically every I/O object in Python has a +file-like wrapper of some sort. Sockets already act as such and for +strings there is StringIO. Popen can be made to act like a file by simply +using the methods attached to the subprocess.Popen.stderr, stdout and +stdin file-like objects. But when using the read and write methods of +those options, you do not have the benefit of asynchronous I/O. In the +proposed solution the wrapper wraps the asynchronous methods to mimic a +file object. - I have been working on implementing non-blocking asynchronous I/O in the - subprocess.Popen module as well as a wrapper class for subprocess.Popen - that makes it so that an executed process can take the place of a file by - duplicating all of the methods and attributes that file objects have. - There are two base functions that have been added to the subprocess.Popen - class: Popen.send and Popen._recv, each with two separate implementations, - one for Windows and one for Unix-based systems. The Windows - implementation uses ctypes to access the functions needed to control pipes - in the kernel 32 DLL in an asynchronous manner. On Unix based systems, - the Python interface for file control serves the same purpose. The - different implementations of Popen.send and Popen._recv have identical - arguments to make code that uses these functions work across multiple - platforms. +Reference Implementation +======================== - When calling the Popen._recv function, it requires the pipe name be - passed as an argument so there exists the Popen.recv function that passes - selects stdout as the pipe for Popen._recv by default. Popen.recv_err - selects stderr as the pipe by default. Popen.recv and Popen.recv_err - are much easier to read and understand than Popen._recv('stdout' ...) and - Popen._recv('stderr' ...) respectively. +I have been maintaining a Google Code repository that contains all of my +changes including tests and documentation [9]_ as well as blog detailing +the problems I have come across in the development process [10]_. - Since the Popen._recv function does not wait on data to be produced - before returning a value, it may return empty bytes. Popen.asyncread - handles this issue by returning all data read over a given time - interval. +I have been working on implementing non-blocking asynchronous I/O in the +subprocess.Popen module as well as a wrapper class for subprocess.Popen +that makes it so that an executed process can take the place of a file by +duplicating all of the methods and attributes that file objects have. - The ProcessIOWrapper class uses the asyncread and asyncwrite functions to - allow a process to act like a file so that there are no blocking issues - that can arise from using the stdout and stdin file objects produced from - a subprocess.Popen call. +There are two base functions that have been added to the subprocess.Popen +class: Popen.send and Popen._recv, each with two separate implementations, +one for Windows and one for Unix-based systems. The Windows +implementation uses ctypes to access the functions needed to control pipes +in the kernel 32 DLL in an asynchronous manner. On Unix based systems, +the Python interface for file control serves the same purpose. The +different implementations of Popen.send and Popen._recv have identical +arguments to make code that uses these functions work across multiple +platforms. +When calling the Popen._recv function, it requires the pipe name be +passed as an argument so there exists the Popen.recv function that passes +selects stdout as the pipe for Popen._recv by default. Popen.recv_err +selects stderr as the pipe by default. Popen.recv and Popen.recv_err +are much easier to read and understand than Popen._recv('stdout' ...) and +Popen._recv('stderr' ...) respectively. -References: +Since the Popen._recv function does not wait on data to be produced +before returning a value, it may return empty bytes. Popen.``asyncread`` +handles this issue by returning all data read over a given time +interval. - [1] [ python-Feature Requests-1191964 ] asynchronous Subprocess - http://mail.python.org/pipermail/python-bugs-list/2006-December/ - 036524.html +The ``ProcessIOWrapper`` class uses the ``asyncread`` and ``asyncwrite`` functions to +allow a process to act like a file so that there are no blocking issues +that can arise from using the stdout and stdin file objects produced from +a subprocess.Popen call. - [2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess - http://ivory.idyll.org/blog/feb-07/problems-with-subprocess - [3] How can I run an external command asynchronously from Python? - Stack - Overflow - http://stackoverflow.com/questions/636561/how-can-i-run-an-external- - command-asynchronously-from-python +References +========== - [4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation - http://docs.python.org/library/subprocess.html#subprocess.Popen.wait +.. [1] [ python-Feature Requests-1191964 ] asynchronous Subprocess + http://mail.python.org/pipermail/python-bugs-list/2006-December/ + 036524.html - [5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation - http://docs.python.org/library/subprocess.html#subprocess.Popen.kill +.. [2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess + http://ivory.idyll.org/blog/feb-07/problems-with-subprocess - [6] Issue 1191964: asynchronous Subprocess - Python tracker - http://bugs.python.org/issue1191964 +.. [3] How can I run an external command asynchronously from Python? - Stack + Overflow + http://stackoverflow.com/questions/636561/how-can-i-run-an-external- + command-asynchronously-from-python - [7] Module to allow Asynchronous subprocess use on Windows and Posix - platforms - ActiveState Code - http://code.activestate.com/recipes/440554/ +.. [4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation + http://docs.python.org/library/subprocess.html#subprocess.Popen.wait - [8] subprocess.rst - subprocdev - Project Hosting on Google Code - http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437 +.. [5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation + http://docs.python.org/library/subprocess.html#subprocess.Popen.kill - [9] subprocdev - Project Hosting on Google Code - http://code.google.com/p/subprocdev +.. [6] Issue 1191964: asynchronous Subprocess - Python tracker + http://bugs.python.org/issue1191964 - [10] Python Subprocess Dev - http://subdev.blogspot.com/ - - [11] https://bugs.python.org/issue18823 -- Idle: use pipes instead of - sockets to talk with user subprocess - -Copyright: - - This P.E.P. is licensed under the Open Publication License; - http://www.opencontent.org/openpub/. +.. [7] Module to allow Asynchronous subprocess use on Windows and Posix + platforms - ActiveState Code + http://code.activestate.com/recipes/440554/ + +.. [8] subprocess.rst - subprocdev - Project Hosting on Google Code + http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437 + +.. [9] subprocdev - Project Hosting on Google Code + http://code.google.com/p/subprocdev + +.. [10] Python Subprocess Dev + http://subdev.blogspot.com/ + +.. [11] https://bugs.python.org/issue18823 -- Idle: use pipes instead of + sockets to talk with user subprocess + +Copyright +========= + +This P.E.P. is licensed under the Open Publication License; +http://www.opencontent.org/openpub/. From e8bdf5664bbe53ca58ed8951c0c476c398c95315 Mon Sep 17 00:00:00 2001 From: Mariatta Date: Mon, 28 Nov 2016 09:08:36 -0800 Subject: [PATCH 3/3] More adjustments remove backticks from Popen.asyncread add missing period. --- pep-3145.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-3145.txt b/pep-3145.txt index f5d58c4da63..e2e57291df6 100644 --- a/pep-3145.txt +++ b/pep-3145.txt @@ -32,7 +32,7 @@ PEP Withdrawal ============== This can be dealt with in the bug tracker. A specific proposal is -attached to [11]_ +attached to [11]_. Motivation @@ -96,7 +96,7 @@ are much easier to read and understand than Popen._recv('stdout' ...) and Popen._recv('stderr' ...) respectively. Since the Popen._recv function does not wait on data to be produced -before returning a value, it may return empty bytes. Popen.``asyncread`` +before returning a value, it may return empty bytes. Popen.asyncread handles this issue by returning all data read over a given time interval.