gh-126925: Modify how iOS test results are gathered

Doc/c-api/init_config.rst

@@ -1281,6 +1281,15 @@ PyConfig
 
       Default: ``1`` in Python config and ``0`` in isolated config.
 
+   .. c:member:: int use_system_logger
+
+      If non-zero, ``stdout`` and ``stderr`` will be redirected to the system
+      log.
+
+      Only available on macOS 10.12 and later, and on iOS.
+
+      Default: ``0`` (don't use system log).
+
    .. c:member:: int user_site_directory
 
       If non-zero, add the user site directory to :data:`sys.path`.

Doc/using/ios.rst

@@ -292,10 +292,12 @@ To add Python to an iOS Xcode project:
 10. Add Objective C code to initialize and use a Python interpreter in embedded
     mode. You should ensure that:
 
-   * :c:member:`UTF-8 mode <PyPreConfig.utf8_mode>` is *enabled*;
-   * :c:member:`Buffered stdio <PyConfig.buffered_stdio>` is *disabled*;
-   * :c:member:`Writing bytecode <PyConfig.write_bytecode>` is *disabled*;
-   * :c:member:`Signal handlers <PyConfig.install_signal_handlers>` are *enabled*;
+   * UTF-8 mode (:c:member:`PyPreConfig.utf8_mode`) is *enabled*;
+   * Buffered stdio (:c:member:`PyConfig.buffered_stdio`) is *disabled*;
+   * Writing bytecode (:c:member:`PyConfig.write_bytecode`) is *disabled*;
+   * Signal handlers (:c:member:`PyConfig.install_signal_handlers`) are *enabled*;
+   * System logging (:c:member:`PyConfig.use_system_logger`) is *enabled*
+     (optional, but strongly recommended);
    * ``PYTHONHOME`` for the interpreter is configured to point at the
      ``python`` subfolder of your app's bundle; and
    * The ``PYTHONPATH`` for the interpreter includes:
@@ -324,6 +326,49 @@ modules in your app, some additional steps will be required:
 * If you're using a separate folder for third-party packages, ensure that folder
   is included as part of the ``PYTHONPATH`` configuration in step 10.
 
+Testing a Python package
+------------------------
+
+The CPython source tree contains :source:`a testbed project <iOS/testbed>` that
+is used to run the CPython test suite on the iOS simulator. This testbed can also
+be used as a testbed project for running your Python library's test suite on iOS.
+
+After building or obtaining an iOS XCFramework (See :source:`iOS/README.rst`
+for details), create a clone of the Python iOS testbed project by running:
+
+.. code-block:: bash
+
+    $ python iOS/testbed clone --framework <path/to/Python.xcframework> --app <path/to/module1> --app <path/to/module2> app-testbed
+
+You will need to modify the ``iOS/testbed`` reference to point to that
+directory in the CPython source tree; any folders specified with the ``--app``
+flag will be copied into the cloned testbed project. The resulting testbed will
+be created in the ``app-testbed`` folder. In this example, the ``module1`` and
+``module2`` would be importable modules at runtime. If your project has
+additional dependencies, they can be installed into the
+``app-testbed/iOSTestbed/app_packages`` folder (using ``pip install --target
+app-testbed/iOSTestbed/app_packages`` or similar).
+
+You can then use the ``app-testbed`` folder to run the test suite for your app,
+For example, if ``module1.tests`` was the entry point to your test suite, you
+could run:
+
+.. code-block:: bash
+
+    $ python app-testbed run -- module1.tests
+
+This is the equivalent of running ``python -m module1.tests`` on a desktop
+Python build. Any arguments after the ``--`` will be passed to the testbed as
+if they were arguments to ``python -m`` on a desktop machine.
+
+You can also open the testbed project in Xcode by running:
+
+.. code-block:: bash
+
+    $ open app-testbed/iOSTestbed.xcodeproj
+
+This will allow you to use the full Xcode suite of tools for debugging.
+
 App Store Compliance
 ====================
 

Include/cpython/initconfig.h

@@ -179,6 +179,9 @@ typedef struct PyConfig {
     int use_frozen_modules;
     int safe_path;
     int int_max_str_digits;
+#ifdef __APPLE__
+    int use_system_logger;
+#endif
 
     int cpu_count;
 #ifdef Py_GIL_DISABLED

Lib/_apple_support.py

@@ -0,0 +1,66 @@
+import io
+import sys
+
+
+def init_streams(log_write, stdout_level, stderr_level):
+    # Redirect stdout and stderr to the Apple system log. This method is
+    # invoked by init_apple_streams() (initconfig.c) if config->use_system_logger
+    # is enabled.
+    sys.stdout = SystemLog(log_write, stdout_level, errors=sys.stderr.errors)
+    sys.stderr = SystemLog(log_write, stderr_level, errors=sys.stderr.errors)
+
+
+class SystemLog(io.TextIOWrapper):
+    def __init__(self, log_write, level, **kwargs):
+        kwargs.setdefault("encoding", "UTF-8")
+        kwargs.setdefault("line_buffering", True)
+        super().__init__(LogStream(log_write, level), **kwargs)
+
+    def __repr__(self):
+        return f"<SystemLog (level {self.buffer.level})>"
+
+    def write(self, s):
+        if not isinstance(s, str):
+            raise TypeError(
+                f"write() argument must be str, not {type(s).__name__}")
+
+        # In case `s` is a str subclass that writes itself to stdout or stderr
+        # when we call its methods, convert it to an actual str.
+        s = str.__str__(s)
+
+        # We want to emit one log message per line, so split
+        # the string before sending it to the superclass.
+        for line in s.splitlines(keepends=True):
+            super().write(line)
+
+        return len(s)
+
+
+class LogStream(io.RawIOBase):
+    def __init__(self, log_write, level):
+        self.log_write = log_write
+        self.level = level
+
+    def __repr__(self):
+        return f"<LogStream (level {self.level!r})>"
+
+    def writable(self):
+        return True
+
+    def write(self, b):
+        if type(b) is not bytes:
+            try:
+                b = bytes(memoryview(b))
+            except TypeError:
+                raise TypeError(
+                    f"write() argument must be bytes-like, not {type(b).__name__}"
+                ) from None
+
+        # Writing an empty string to the stream should have no effect.
+        if b:
+            # Encode null bytes using "modified UTF-8" to avoid truncating the
+            # message. This should not affect the return value, as the caller
+            # may be expecting it to match the length of the input.
+            self.log_write(self.level, b.replace(b"\x00", b"\xc0\x80"))
+
+        return len(b)

Lib/test/test_apple.py

@@ -0,0 +1,155 @@
+import unittest
+from _apple_support import SystemLog
+from test.support import is_apple
+from unittest.mock import Mock, call
+
+if not is_apple:
+    raise unittest.SkipTest("Apple-specific")
+
+
+# Test redirection of stdout and stderr to the Android log.
+class TestAppleSystemLogOutput(unittest.TestCase):
+    maxDiff = None
+
+    def assert_writes(self, output):
+        self.assertEqual(
+            self.log_write.mock_calls,
+            [
+                call(self.log_level, line)
+                for line in output
+            ]
+        )
+
+        self.log_write.reset_mock()
+
+    def setUp(self):
+        self.log_write = Mock()
+        self.log_level = 42
+        self.log = SystemLog(self.log_write, self.log_level, errors="replace")
+
+    def test_repr(self):
+        self.assertEqual(repr(self.log), "<SystemLog (level 42)>")
+        self.assertEqual(repr(self.log.buffer), "<LogStream (level 42)>")
+
+    def test_log_config(self):
+        self.assertIs(self.log.writable(), True)
+        self.assertIs(self.log.readable(), False)
+
+        self.assertEqual("UTF-8", self.log.encoding)
+        self.assertEqual("replace", self.log.errors)
+
+        self.assertIs(self.log.line_buffering, True)
+        self.assertIs(self.log.write_through, False)
+
+    def test_empty_str(self):
+        self.log.write("")
+        self.log.flush()
+
+        self.assert_writes([])
+
+    def test_simple_str(self):
+        self.log.write("hello world\n")
+
+        self.assert_writes([b"hello world\n"])
+
+    def test_buffered_str(self):
+        self.log.write("h")
+        self.log.write("ello")
+        self.log.write(" ")
+        self.log.write("world\n")
+        self.log.write("goodbye.")
+        self.log.flush()
+
+        self.assert_writes([b"hello world\n", b"goodbye."])
+
+    def test_manual_flush(self):
+        self.log.write("Hello")
+
+        self.assert_writes([])
+
+        self.log.write(" world\nHere for a while...\nGoodbye")
+        self.assert_writes([b"Hello world\n", b"Here for a while...\n"])
+
+        self.log.write(" world\nHello again")
+        self.assert_writes([b"Goodbye world\n"])
+
+        self.log.flush()
+        self.assert_writes([b"Hello again"])
+
+    def test_non_ascii(self):
+        # Spanish
+        self.log.write("ol\u00e9\n")
+        self.assert_writes([b"ol\xc3\xa9\n"])
+
+        # Chinese
+        self.log.write("\u4e2d\u6587\n")
+        self.assert_writes([b"\xe4\xb8\xad\xe6\x96\x87\n"])
+
+        # Printing Non-BMP emoji
+        self.log.write("\U0001f600\n")
+        self.assert_writes([b"\xf0\x9f\x98\x80\n"])
+
+        # Non-encodable surrogates are replaced
+        self.log.write("\ud800\udc00\n")
+        self.assert_writes([b"??\n"])
+
+    def test_modified_null(self):
+        # Null characters are logged using "modified UTF-8".
+        self.log.write("\u0000\n")
+        self.assert_writes([b"\xc0\x80\n"])
+        self.log.write("a\u0000\n")
+        self.assert_writes([b"a\xc0\x80\n"])
+        self.log.write("\u0000b\n")
+        self.assert_writes([b"\xc0\x80b\n"])
+        self.log.write("a\u0000b\n")
+        self.assert_writes([b"a\xc0\x80b\n"])
+
+    def test_nonstandard_str(self):
+        # String subclasses are accepted, but they should be converted
+        # to a standard str without calling any of their methods.
+        class CustomStr(str):
+            def splitlines(self, *args, **kwargs):
+                raise AssertionError()
+
+            def __len__(self):
+                raise AssertionError()
+
+            def __str__(self):
+                raise AssertionError()
+
+        self.log.write(CustomStr("custom\n"))
+        self.assert_writes([b"custom\n"])
+
+    def test_non_str(self):
+        # Non-string classes are not accepted.
+        for obj in [b"", b"hello", None, 42]:
+            with self.subTest(obj=obj):
+                with self.assertRaisesRegex(
+                    TypeError,
+                    fr"write\(\) argument must be str, not "
+                    fr"{type(obj).__name__}"
+                ):
+                    self.log.write(obj)
+
+    def test_byteslike_in_buffer(self):
+        # The underlying buffer *can* accept bytes-like objects
+        self.log.buffer.write(bytearray(b"hello"))
+        self.log.flush()
+
+        self.log.buffer.write(b"")
+        self.log.flush()
+
+        self.log.buffer.write(b"goodbye")
+        self.log.flush()
+
+        self.assert_writes([b"hello", b"goodbye"])
+
+    def test_non_byteslike_in_buffer(self):
+        for obj in ["hello", None, 42]:
+            with self.subTest(obj=obj):
+                with self.assertRaisesRegex(
+                    TypeError,
+                    fr"write\(\) argument must be bytes-like, not "
+                    fr"{type(obj).__name__}"
+                ):
+                    self.log.buffer.write(obj)

Lib/test/test_capi/test_config.py

@@ -110,6 +110,10 @@ def test_config_get(self):
             options.extend((
                 ("_pystats", bool, None),
             ))
+        if support.is_apple:
+            options.extend((
+                ("use_system_logger", bool, None),
+            ))
 
         for name, option_type, sys_attr in options:
             with self.subTest(name=name, option_type=option_type,

Lib/test/test_embed.py

@@ -649,6 +649,8 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase):
         CONFIG_COMPAT.update({
             'legacy_windows_stdio': False,
         })
+    if support.is_apple:
+        CONFIG_COMPAT['use_system_logger'] = False
 
     CONFIG_PYTHON = dict(CONFIG_COMPAT,
         _config_init=API_PYTHON,

Makefile.pre.in

@@ -2146,7 +2146,6 @@ testuniversal: all
 # This must be run *after* a `make install` has completed the build. The
 # `--with-framework-name` argument *cannot* be used when configuring the build.
 XCFOLDER:=iOSTestbed.$(MULTIARCH).$(shell date +%s)
-XCRESULT=$(XCFOLDER)/$(MULTIARCH).xcresult
 .PHONY: testios
 testios:
 	@if test "$(MACHDEP)" != "ios"; then \
@@ -2165,29 +2164,12 @@ testios:
 		echo "Cannot find a finalized iOS Python.framework. Have you run 'make install' to finalize the framework build?"; \
 		exit 1;\
 	fi
-	# Copy the testbed project into the build folder
-	cp -r $(srcdir)/iOS/testbed $(XCFOLDER)
-	# Copy the framework from the install location to the testbed project.
-	cp -r $(PYTHONFRAMEWORKPREFIX)/* $(XCFOLDER)/Python.xcframework/ios-arm64_x86_64-simulator
-
-	# Run the test suite for the Xcode project, targeting the iOS simulator.
-	# If the suite fails, touch a file in the test folder as a marker
-	if ! xcodebuild test -project $(XCFOLDER)/iOSTestbed.xcodeproj -scheme "iOSTestbed" -destination "platform=iOS Simulator,name=iPhone SE (3rd Generation)" -resultBundlePath $(XCRESULT) -derivedDataPath $(XCFOLDER)/DerivedData ; then \
-	 	touch $(XCFOLDER)/failed; \
-	fi
 
-	# Regardless of success or failure, extract and print the test output
-	xcrun xcresulttool get --path $(XCRESULT) \
-		--id $$( \
-			xcrun xcresulttool get --path $(XCRESULT) --format json | \
-			$(PYTHON_FOR_BUILD) -c "import sys, json; result = json.load(sys.stdin); print(result['actions']['_values'][0]['actionResult']['logRef']['id']['_value'])" \
-		) \
-		--format json | \
-		$(PYTHON_FOR_BUILD) -c "import sys, json; result = json.load(sys.stdin); print(result['subsections']['_values'][1]['subsections']['_values'][0]['emittedOutput']['_value'])"
+	# Clone the testbed project into the XCFOLDER
+	$(PYTHON_FOR_BUILD) $(srcdir)/iOS/testbed clone --framework $(PYTHONFRAMEWORKPREFIX) "$(XCFOLDER)"
 
-	@if test -e $(XCFOLDER)/failed ; then \
-		exit 1; \
-	fi
+	# Run the testbed project
+	$(PYTHON_FOR_BUILD) "$(XCFOLDER)" run -- test -uall --single-process --rerun -W
 
 # Like test, but using --slow-ci which enables all test resources and use
 # longer timeout. Run an optional pybuildbot.identify script to include

Misc/NEWS.d/next/Library/2024-12-04-15-04-12.gh-issue-126821.lKCLVV.rst

@@ -0,0 +1,2 @@
+macOS and iOS apps can now choose to redirect stdout and stderr to the
+system log during interpreter configuration.

Misc/NEWS.d/next/Tests/2024-12-04-15-03-24.gh-issue-126925.uxAMK-.rst

@@ -0,0 +1,2 @@
+iOS test results are now streamed during test execution, and the deprecated
+xcresulttool is no longer used.