Skip to content

Commit 5b39852

Browse files
miss-islingtonvsajip
authored andcommitted
Update logging cookbook to show multiple worker processes using the concurrent.futures module. (GH-14905) (GH-14906)
(cherry picked from commit d309352)
1 parent 22fd679 commit 5b39852

File tree

1 file changed

+35
-0
lines changed

1 file changed

+35
-0
lines changed

Doc/howto/logging-cookbook.rst

+35
Original file line numberDiff line numberDiff line change
@@ -948,6 +948,41 @@ This variant shows how you can e.g. apply configuration for particular loggers
948948
machinery in the main process (even though the logging events are generated in
949949
the worker processes) to direct the messages to the appropriate destinations.
950950

951+
Using concurrent.futures.ProcessPoolExecutor
952+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
953+
954+
If you want to use :class:`concurrent.futures.ProcessPoolExecutor` to start
955+
your worker processes, you need to create the queue slightly differently.
956+
Instead of
957+
958+
.. code-block:: python
959+
960+
queue = multiprocessing.Queue(-1)
961+
962+
you should use
963+
964+
.. code-block:: python
965+
966+
queue = multiprocessing.Manager().Queue(-1) # also works with the examples above
967+
968+
and you can then replace the worker creation from this::
969+
970+
workers = []
971+
for i in range(10):
972+
worker = multiprocessing.Process(target=worker_process,
973+
args=(queue, worker_configurer))
974+
workers.append(worker)
975+
worker.start()
976+
for w in workers:
977+
w.join()
978+
979+
to this (remembering to first import :mod:`concurrent.futures`)::
980+
981+
with concurrent.futures.ProcessPoolExecutor(max_workers=10) as executor:
982+
for i in range(10):
983+
executor.submit(worker_process, queue, worker_configurer)
984+
985+
951986
Using file rotation
952987
-------------------
953988

0 commit comments

Comments
 (0)