From c74597703d1cf5de4ab028df855dacdd382c966c Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Sun, 1 Oct 2023 13:25:07 +0200 Subject: [PATCH 01/18] gh-107954, PEP 741: Add PyInitConfig C API Add Doc/c-api/config.rst documentation. --- Doc/c-api/config.rst | 227 +++++++++ Doc/c-api/index.rst | 1 + Doc/c-api/init.rst | 3 +- Doc/c-api/init_config.rst | 3 +- Doc/whatsnew/3.14.rst | 18 + Include/cpython/initconfig.h | 60 +++ Include/internal/pycore_initconfig.h | 4 + Lib/test/test_embed.py | 43 +- ...-08-30-14-02-17.gh-issue-107954.TPvj4u.rst | 17 + Objects/unicodeobject.c | 4 +- Programs/_testembed.c | 84 +++ Python/initconfig.c | 478 +++++++++++++++++- Python/preconfig.c | 52 +- 13 files changed, 952 insertions(+), 42 deletions(-) create mode 100644 Doc/c-api/config.rst create mode 100644 Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst diff --git a/Doc/c-api/config.rst b/Doc/c-api/config.rst new file mode 100644 index 00000000000000..b795dc37e7024f --- /dev/null +++ b/Doc/c-api/config.rst @@ -0,0 +1,227 @@ +.. highlight:: c + +.. _config-c-api: + +******************** +Python Configuration +******************** + +.. versionadded:: 3.14 + +C API to configure the Python initialization (:pep:`741`). + +See also :ref:`Python Initialization Configuration `. + + +Initialize Python +================= + +Create Config +------------- + +.. c:struct:: PyInitConfig + + Opaque structure to configure the Python initialization. + + +.. c:function:: PyInitConfig* PyInitConfig_Create(void) + + Create a new initialization configuration using :ref:`Isolated Configuration + ` default values. + + It must be freed by :c:func:`PyInitConfig_Free`. + + Return ``NULL`` on memory allocation failure. + + +.. c:function:: void PyInitConfig_Free(PyInitConfig *config) + + Free memory of the initialization configuration *config*. + + +Get Options +----------- + +The configuration option *name* parameter must be a non-NULL +null-terminated UTF-8 encoded string. + +.. c:function:: int PyInitConfig_HasOption(PyInitConfig *config, const char *name) + + Test if the configuration has an option called *name*. + + Return ``1`` if the option exists, or return ``0`` otherwise. + + +.. c:function:: int PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) + + Get an integer configuration option. + + * Set *\*value*, and return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +.. c:function:: int PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) + + Get a string configuration option as a null-terminated UTF-8 + encoded string. + + * Set *\*value*, and return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + On success, the string must be released with ``free(value)``. + + +.. c:function:: int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) + + Get a string list configuration option as an array of + null-terminated UTF-8 encoded strings. + + * Set *\*length* and *\*value*, and return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + On success, the string list must be released with + ``PyInitConfig_FreeStrList(length, items)``. + + +.. c:function:: void PyInitConfig_FreeStrList(size_t length, char **items) + + Free memory of a string list created by + ``PyInitConfig_GetStrList()``. + + +Set Options +----------- + +The configuration option *name* parameter must be a non-NULL null-terminated +UTF-8 encoded string. + +Some configuration options have side effects on other options. This logic is +only implemented when ``Py_InitializeFromInitConfig()`` is called, not by the +"Set" functions below. For example, setting ``dev_mode`` to ``1`` does not set +``faulthandler`` to ``1``. + +.. c:function:: int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) + + Set an integer configuration option. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +.. c:function:: int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char *value) + + Set a string configuration option from a null-terminated UTF-8 + encoded string. The string is copied. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +.. c:function:: int PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items) + + Set a string list configuration option from an array of + null-terminated UTF-8 encoded strings. The string list is copied. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +Initialize Python +----------------- + +.. c:function:: int Py_InitializeFromInitConfig(PyInitConfig *config) + + Initialize Python from the initialization configuration. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + * Set an exit code in *config* and return ``-1`` if Python wants to + exit. + + See ``PyInitConfig_GetExitcode()`` for the exit code case. + + +Error Handling +-------------- + +.. c:function:: int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg) + + Get the *config* error message. + + * Set *\*err_msg* and return ``1`` if an error is set. + * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise. + + An error message is an UTF-8 encoded string. + + If *config* has an exit code, format the exit code as an error + message. + + The error message remains valid until another ``PyInitConfig`` + function is called with *config*. The caller doesn't have to free the + error message. + + +.. c:function:: int PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) + + Get the *config* exit code. + + * Set *\*exitcode* and return ``1`` if Python wants to exit. + * Return ``0`` if *config* has no exit code set. + + Only the ``Py_InitializeFromInitConfig()`` function can set an exit + code if the ``parse_argv`` option is non-zero. + + An exit code can be set when parsing the command line failed (exit + code ``2``) or when a command line option asks to display the command + line help (exit code ``0``). + + +Example +======= + +Example initializing Python, set configuration options of various types, +return ``-1`` on error: + +.. code-block:: c + + int init_python(void) + { + PyInitConfig *config = PyInitConfig_Create(); + if (config == NULL) { + printf("PYTHON INIT ERROR: memory allocation failed\n"); + return -1; + } + + // Set an integer (dev mode) + if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) { + goto error; + } + + // Set a list of UTF-8 strings (argv) + char *argv[] = {"my_program", "-c", "pass"}; + if (PyInitConfig_SetStrList(config, "argv", + Py_ARRAY_LENGTH(argv), argv) < 0) { + goto error; + } + + // Set a UTF-8 string (program name) + if (PyInitConfig_SetStr(config, "program_name", L"my_program") < 0) { + goto error; + } + + // Initialize Python with the configuration + if (Py_InitializeFromInitConfig(config) < 0) { + goto error; + } + PyInitConfig_Free(config); + return 0; + + error: + // Display the error message + const char *err_msg; + (void)PyInitConfig_GetError(config, &err_msg); + printf("PYTHON INIT ERROR: %s\n", err_msg); + PyInitConfig_Free(config); + + return -1; + } diff --git a/Doc/c-api/index.rst b/Doc/c-api/index.rst index ba56b03c6ac8e7..fa2ada706072aa 100644 --- a/Doc/c-api/index.rst +++ b/Doc/c-api/index.rst @@ -22,6 +22,7 @@ document the API functions in detail. concrete.rst init.rst init_config.rst + config.rst memory.rst objimpl.rst apiabiversion.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 0965fd97e09113..4147deff4f839d 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -7,7 +7,8 @@ Initialization, Finalization, and Threads ***************************************** -See also :ref:`Python Initialization Configuration `. +See also the :ref:`Python Initialization Configuration ` +and the :ref:`Python Configuration ` .. _pre-init-safe: diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 6e2e04fba55a45..59c33b842a3a38 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -27,7 +27,8 @@ There are two kinds of configuration: The :c:func:`Py_RunMain` function can be used to write a customized Python program. -See also :ref:`Initialization, Finalization, and Threads `. +See also :ref:`Initialization, Finalization, and Threads ` +and the :ref:`Python Configuration `. .. seealso:: :pep:`587` "Python Initialization Configuration". diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 975af420f9b375..6d8ee8bb2a3dae 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -489,6 +489,24 @@ New Features similar to ``sep.join(iterable)`` in Python. (Contributed by Victor Stinner in :gh:`121645`.) +* Add functions to configure the Python initialization: + + * :c:func:`PyInitConfig_Create` + * :c:func:`PyInitConfig_Free` + * :c:func:`PyInitConfig_HasOption` + * :c:func:`PyInitConfig_GetInt` + * :c:func:`PyInitConfig_GetStr` + * :c:func:`PyInitConfig_GetStrList` + * :c:func:`PyInitConfig_FreeStrList` + * :c:func:`PyInitConfig_SetInt` + * :c:func:`PyInitConfig_SetStr` + * :c:func:`PyInitConfig_SetStrList` + * :c:func:`Py_InitializeFromInitConfig` + * :c:func:`PyInitConfig_GetError` + * :c:func:`PyInitConfig_GetExitCode` + + (Contributed by Victor Stinner in :gh:`107954`.) + Porting to Python 3.14 ---------------------- diff --git a/Include/cpython/initconfig.h b/Include/cpython/initconfig.h index 5da5ef9e5431b1..035a0dc92fa94c 100644 --- a/Include/cpython/initconfig.h +++ b/Include/cpython/initconfig.h @@ -267,6 +267,66 @@ PyAPI_FUNC(PyStatus) PyConfig_SetWideStringList(PyConfig *config, See also PyConfig.orig_argv. */ PyAPI_FUNC(void) Py_GetArgcArgv(int *argc, wchar_t ***argv); + +// --- PyInitConfig --------------------------------------------------------- + +typedef struct PyInitConfig PyInitConfig; + +// Create a new initialization configuration using the Isolated configuration +// defaults. +// It must be freed by PyInitConfig_Free(). +// Return NULL on memory allocation failure. +PyAPI_FUNC(PyInitConfig*) PyInitConfig_Create(void); + +// Free memory of a initialization configuration. +PyAPI_FUNC(void) PyInitConfig_Free(PyInitConfig *config); + +// Set an integer configuration option. +// Return 0 on success, or return -1 on error. +PyAPI_FUNC(int) PyInitConfig_SetInt( + PyInitConfig *config, + const char *name, + int64_t value); + +// Set a string configuration option from a bytes string. +// +// The bytes string is decoded by Py_DecodeLocale(). Preinitialize Python if +// needed to ensure that encodings are properly configured. +// +// Return 0 on success, or return -1 on error. +PyAPI_FUNC(int) PyInitConfig_SetStr( + PyInitConfig *config, + const char *name, + const char *value); + +// Set a string list configuration option from bytes strings. +// +// The bytes strings are decoded by Py_DecodeLocale(). Preinitialize Python if +// needed to ensure that encodings are properly configured. +// +// Return 0 on success, or return -1 on error. +PyAPI_FUNC(int) PyInitConfig_SetStrList( + PyInitConfig *config, + const char *name, + size_t length, + char * const *items); + +// Initialize Python from the initialization configuration. +// Return 0 on success. +// Return -1 if Python wants to exit and on error +PyAPI_FUNC(int) Py_InitializeFromInitConfig(PyInitConfig *config); + +// Get the error message. +// Set *err_msg and return 1 if an error is set. +// Set *err_msg to NULL and return 0 otherwise. +PyAPI_FUNC(int) PyInitConfig_GetError(PyInitConfig* config, const char **err_msg); + +// Get the exit code. +// Set '*exitcode' and return 1 if an exit code is set. +// Return 0 otherwise. +PyAPI_FUNC(int) PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode); + + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_initconfig.h b/Include/internal/pycore_initconfig.h index 6bf1b53bffd3ba..2c58b501e3369c 100644 --- a/Include/internal/pycore_initconfig.h +++ b/Include/internal/pycore_initconfig.h @@ -62,6 +62,10 @@ extern int _PyWideStringList_Copy(PyWideStringList *list, extern PyStatus _PyWideStringList_Extend(PyWideStringList *list, const PyWideStringList *list2); extern PyObject* _PyWideStringList_AsList(const PyWideStringList *list); +extern PyStatus _PyWideStringList_FromBytes( + PyWideStringList *list, + Py_ssize_t length, + char * const *items); /* --- _PyArgv ---------------------------------------------------- */ diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 7860c67f082b1f..4bcbe76bdbbaf6 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -494,6 +494,14 @@ def test_getargs_reset_static_parser(self): self.assertEqual(out, '1\n2\n3\n' * INIT_LOOPS) +def config_dev_mode(preconfig, config): + preconfig['allocator'] = PYMEM_ALLOCATOR_DEBUG + preconfig['dev_mode'] = 1 + config['dev_mode'] = 1 + config['warnoptions'] = ['default'] + config['faulthandler'] = 1 + + @unittest.skipIf(_testinternalcapi is None, "requires _testinternalcapi") class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): maxDiff = 4096 @@ -1102,23 +1110,20 @@ def test_init_env_dev_mode_alloc(self): api=API_COMPAT) def test_init_dev_mode(self): - preconfig = { - 'allocator': PYMEM_ALLOCATOR_DEBUG, - } + preconfig = {} config = { 'faulthandler': True, 'dev_mode': True, 'warnoptions': ['default'], } + config_dev_mode(preconfig, config) self.check_all_configs("test_init_dev_mode", config, preconfig, api=API_PYTHON) def test_preinit_parse_argv(self): # Pre-initialize implicitly using argv: make sure that -X dev # is used to configure the allocation in preinitialization - preconfig = { - 'allocator': PYMEM_ALLOCATOR_DEBUG, - } + preconfig = {} config = { 'argv': ['script.py'], 'orig_argv': ['python3', '-X', 'dev', '-P', 'script.py'], @@ -1129,6 +1134,7 @@ def test_preinit_parse_argv(self): 'xoptions': ['dev'], 'safe_path': True, } + config_dev_mode(preconfig, config) self.check_all_configs("test_preinit_parse_argv", config, preconfig, api=API_PYTHON) @@ -1728,16 +1734,15 @@ def test_init_warnoptions(self): 'ignore:::PySys_AddWarnOption2', # PySys_AddWarnOption() 'ignore:::PyConfig_BeforeRead', # PyConfig.warnoptions 'ignore:::PyConfig_AfterRead'] # PyWideStringList_Append() - preconfig = dict(allocator=PYMEM_ALLOCATOR_DEBUG) + preconfig = {} config = { - 'dev_mode': 1, - 'faulthandler': 1, 'bytes_warning': 1, - 'warnoptions': warnoptions, 'orig_argv': ['python3', '-Wignore:::cmdline1', '-Wignore:::cmdline2'], } + config_dev_mode(preconfig, config) + config['warnoptions'] = warnoptions self.check_all_configs("test_init_warnoptions", config, preconfig, api=API_PYTHON) @@ -1750,6 +1755,24 @@ def test_init_set_config(self): self.check_all_configs("test_init_set_config", config, api=API_ISOLATED) + def test_initconfig_api(self): + preconfig = {} + config = { + 'pycache_prefix': 'conf_pycache_prefix', + 'xoptions': ['faulthandler'], + 'hash_seed': 10, + 'use_hash_seed': True, + } + preconfig['allocator'] = PYMEM_ALLOCATOR_DEBUG + preconfig['dev_mode'] = 1 + config['dev_mode'] = 1 + config['warnoptions'] = ['default'] + self.check_all_configs("test_initconfig_api", config, preconfig, + api=API_ISOLATED) + + def test_initconfig_exit(self): + self.run_embedded_interpreter("test_initconfig_exit") + def test_get_argc_argv(self): self.run_embedded_interpreter("test_get_argc_argv") # ignore output diff --git a/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst b/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst new file mode 100644 index 00000000000000..6caff79d2b3a1c --- /dev/null +++ b/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst @@ -0,0 +1,17 @@ +Add functions to configure the Python initialization: + +* :c:func:`PyInitConfig_Create` +* :c:func:`PyInitConfig_Free` +* :c:func:`PyInitConfig_HasOption` +* :c:func:`PyInitConfig_GetInt` +* :c:func:`PyInitConfig_GetStr` +* :c:func:`PyInitConfig_GetStrList` +* :c:func:`PyInitConfig_FreeStrList` +* :c:func:`PyInitConfig_SetInt` +* :c:func:`PyInitConfig_SetStr` +* :c:func:`PyInitConfig_SetStrList` +* :c:func:`Py_InitializeFromInitConfig` +* :c:func:`PyInitConfig_GetError` +* :c:func:`PyInitConfig_GetExitCode` + +Patch by Victor Stinner. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index f4239cee8f7662..d7672d36e8e13d 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -5393,7 +5393,7 @@ _Py_DecodeUTF8_surrogateescape(const char *arg, Py_ssize_t arglen, } -/* UTF-8 encoder using the surrogateescape error handler . +/* UTF-8 encoder. On success, return 0 and write the newly allocated character string (use PyMem_Free() to free the memory) into *str. @@ -15906,7 +15906,7 @@ encode_wstr_utf8(wchar_t *wstr, char **str, const char *name) int res; res = _Py_EncodeUTF8Ex(wstr, str, NULL, NULL, 1, _Py_ERROR_STRICT); if (res == -2) { - PyErr_Format(PyExc_RuntimeWarning, "cannot decode %s", name); + PyErr_Format(PyExc_RuntimeWarning, "cannot encode %s", name); return -1; } if (res < 0) { diff --git a/Programs/_testembed.c b/Programs/_testembed.c index e341f0c6bfc595..b168c12f26b6b1 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -36,6 +36,7 @@ char **main_argv; /* Use path starting with "./" avoids a search along the PATH */ #define PROGRAM_NAME L"./_testembed" +#define PROGRAM_NAME_UTF8 "./_testembed" #define INIT_LOOPS 4 @@ -1805,6 +1806,87 @@ static int test_init_set_config(void) } +static int test_initconfig_api(void) +{ + PyInitConfig *config = PyInitConfig_Create(); + if (config == NULL) { + printf("Init allocation error\n"); + return 1; + } + + if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) { + goto error; + } + + if (PyInitConfig_SetInt(config, "hash_seed", 10) < 0) { + goto error; + } + + // Set a UTF-8 string (program_name) + if (PyInitConfig_SetStr(config, "program_name", PROGRAM_NAME_UTF8) < 0) { + goto error; + } + + // Set a UTF-8 string (pycache_prefix) + if (PyInitConfig_SetStr(config, "pycache_prefix", + "conf_pycache_prefix") < 0) { + goto error; + } + + // Set a list of UTF-8 strings (argv) + char* xoptions[] = {"faulthandler"}; + if (PyInitConfig_SetStrList(config, "xoptions", + Py_ARRAY_LENGTH(xoptions), xoptions) < 0) { + goto error; + } + + + if (Py_InitializeFromInitConfig(config) < 0) { + goto error; + } + PyInitConfig_Free(config); + + dump_config(); + Py_Finalize(); + return 0; + +error: + const char *err_msg; + (void)PyInitConfig_GetError(config, &err_msg); + printf("Python init failed: %s\n", err_msg); + exit(1); +} + + +static int test_initconfig_exit(void) +{ + PyInitConfig *config = PyInitConfig_Create(); + if (config == NULL) { + printf("Init allocation error\n"); + return 1; + } + + char *argv[] = {PROGRAM_NAME_UTF8, "--help"}; + assert(PyInitConfig_SetStrList(config, "argv", + Py_ARRAY_LENGTH(argv), argv) == 0); + + assert(PyInitConfig_SetInt(config, "parse_argv", 1) == 0); + + assert(Py_InitializeFromInitConfig(config) < 0); + + int exitcode; + assert(PyInitConfig_GetExitCode(config, &exitcode) == 1); + assert(exitcode == 0); + + const char *err_msg; + assert(PyInitConfig_GetError(config, &err_msg) == 1); + assert(strcmp(err_msg, "exit code 0") == 0); + + PyInitConfig_Free(config); + return 0; +} + + static void configure_init_main(PyConfig *config) { wchar_t* argv[] = { @@ -2223,6 +2305,8 @@ static struct TestCase TestCases[] = { {"test_init_is_python_build", test_init_is_python_build}, {"test_init_warnoptions", test_init_warnoptions}, {"test_init_set_config", test_init_set_config}, + {"test_initconfig_api", test_initconfig_api}, + {"test_initconfig_exit", test_initconfig_exit}, {"test_run_main", test_run_main}, {"test_run_main_loop", test_run_main_loop}, {"test_get_argc_argv", test_get_argc_argv}, diff --git a/Python/initconfig.c b/Python/initconfig.c index 51897a2d0aef66..6e0ae7b0fbc2a5 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -26,6 +26,8 @@ #include "config_common.h" +// Forward declarations +typedef struct PyConfigSpec PyConfigSpec; /* --- PyConfig spec ---------------------------------------------- */ @@ -40,11 +42,11 @@ typedef enum { PyConfig_MEMBER_WSTR_LIST = 12, } PyConfigMemberType; -typedef struct { +struct PyConfigSpec { const char *name; size_t offset; PyConfigMemberType type; -} PyConfigSpec; +}; #define SPEC(MEMBER, TYPE) \ {#MEMBER, offsetof(PyConfig, MEMBER), PyConfig_MEMBER_##TYPE} @@ -990,7 +992,7 @@ config_set_bytes_string(PyConfig *config, wchar_t **config_str, /* Decode str using Py_DecodeLocale() and set the result into *config_str. - Pre-initialize Python if needed to ensure that encodings are properly + Preinitialize Python if needed to ensure that encodings are properly configured. */ PyStatus PyConfig_SetBytesString(PyConfig *config, wchar_t **config_str, @@ -1000,6 +1002,13 @@ PyConfig_SetBytesString(PyConfig *config, wchar_t **config_str, } +static inline void* +config_spec_get_member(const PyConfigSpec *spec, const PyConfig *config) +{ + return (char *)config + spec->offset; +} + + PyStatus _PyConfig_Copy(PyConfig *config, const PyConfig *config2) { @@ -1008,19 +1017,19 @@ _PyConfig_Copy(PyConfig *config, const PyConfig *config2) PyStatus status; const PyConfigSpec *spec = PYCONFIG_SPEC; for (; spec->name != NULL; spec++) { - char *member = (char *)config + spec->offset; - char *member2 = (char *)config2 + spec->offset; + char *member = config_spec_get_member(spec, config); + const char *member2 = config_spec_get_member(spec, (PyConfig*)config2); switch (spec->type) { case PyConfig_MEMBER_INT: case PyConfig_MEMBER_UINT: case PyConfig_MEMBER_BOOL: { - *(int*)member = *(int*)member2; + *(int*)member = *(const int*)member2; break; } case PyConfig_MEMBER_ULONG: { - *(unsigned long*)member = *(unsigned long*)member2; + *(unsigned long*)member = *(const unsigned long*)member2; break; } case PyConfig_MEMBER_WSTR: @@ -1059,7 +1068,8 @@ _PyConfig_AsDict(const PyConfig *config) const PyConfigSpec *spec = PYCONFIG_SPEC; for (; spec->name != NULL; spec++) { - char *member = (char *)config + spec->offset; + char *member = config_spec_get_member(spec, config); + PyObject *obj; switch (spec->type) { case PyConfig_MEMBER_INT: @@ -1107,6 +1117,7 @@ _PyConfig_AsDict(const PyConfig *config) Py_DECREF(dict); return NULL; } + int res = PyDict_SetItemString(dict, spec->name, obj); Py_DECREF(obj); if (res < 0) { @@ -1273,7 +1284,7 @@ _PyConfig_FromDict(PyConfig *config, PyObject *dict) const PyConfigSpec *spec = PYCONFIG_SPEC; for (; spec->name != NULL; spec++) { - char *member = (char *)config + spec->offset; + char *member = config_spec_get_member(spec, config); switch (spec->type) { case PyConfig_MEMBER_INT: case PyConfig_MEMBER_UINT: @@ -3261,3 +3272,452 @@ _Py_DumpPathConfig(PyThreadState *tstate) _PyErr_SetRaisedException(tstate, exc); } + + +// --- PyInitConfig API --------------------------------------------------- + +struct PyInitConfig { + // FIXME: add preconfig + PyConfig config; + PyStatus status; + char *err_msg; +}; + +static PyInitConfig* +initconfig_alloc(void) +{ + return calloc(1, sizeof(PyInitConfig)); +} + + +PyInitConfig* +PyInitConfig_Create(void) +{ + PyInitConfig *config = initconfig_alloc(); + if (config == NULL) { + return NULL; + } + PyConfig_InitIsolatedConfig(&config->config); + config->status = _PyStatus_OK(); + return config; +} + + +void +PyInitConfig_Free(PyInitConfig *config) +{ + free(config->err_msg); + free(config); +} + + +static void +initconfig_set_error(PyInitConfig *config, const char *err_msg) +{ + config->status = _PyStatus_ERR(err_msg); +} + + +static const PyConfigSpec* +config_get_spec(const char *name) +{ + const PyConfigSpec *spec = PYCONFIG_SPEC; + for (; spec->name != NULL; spec++) { + if (strcmp(name, spec->name) == 0) { + return spec; + } + } + return NULL; +} + + +int +PyInitConfig_HasOption(PyInitConfig *config, const char *name) +{ + const PyConfigSpec *spec = config_get_spec(name); + return (spec != NULL); +} + + +static const PyConfigSpec* +initconfig_prepare_get(PyInitConfig *config, const char *name) +{ + const PyConfigSpec *spec = config_get_spec(name); + if (spec == NULL) { + initconfig_set_error(config, "unknown config option name"); + return NULL; + } + return spec; +} + + +int +PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) +{ + const PyConfigSpec *spec = initconfig_prepare_get(config, name); + if (spec == NULL) { + return -1; + } + + switch (spec->type) { + case PyConfig_MEMBER_INT: + case PyConfig_MEMBER_UINT: + case PyConfig_MEMBER_BOOL: + { + int *member = config_spec_get_member(spec, &config->config); + *value = *member; + break; + } + + case PyConfig_MEMBER_ULONG: + { + unsigned long *member = config_spec_get_member(spec, &config->config); +#if SIZEOF_LONG >= 8 + if ((unsigned long)INT64_MAX < *member) { + initconfig_set_error(config, + "config option value doesn't fit into int64_t"); + return -1; + } +#endif + *value = *member; + break; + } + + default: + initconfig_set_error(config, "config option type is not int"); + return -1; + } + return 0; +} + + +static char* +wstr_to_utf8(PyInitConfig *config, wchar_t *wstr) +{ + char *utf8; + int res = _Py_EncodeUTF8Ex(wstr, &utf8, NULL, NULL, 1, _Py_ERROR_STRICT); + if (res == -2) { + initconfig_set_error(config, "encoding error"); + return NULL; + } + if (res < 0) { + config->status = _PyStatus_NO_MEMORY(); + return NULL; + } + + // Copy to use the malloc() memory allocator + size_t size = strlen(utf8) + 1; + char *str = malloc(size); + if (str == NULL) { + PyMem_RawFree(utf8); + config->status = _PyStatus_NO_MEMORY(); + return NULL; + } + + memcpy(str, utf8, size); + PyMem_RawFree(utf8); + return str; +} + + +int +PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) +{ + const PyConfigSpec *spec = initconfig_prepare_get(config, name); + if (spec == NULL) { + return -1; + } + + if (spec->type != PyConfig_MEMBER_WSTR + && spec->type != PyConfig_MEMBER_WSTR_OPT) { + initconfig_set_error(config, "config option type is not string"); + return -1; + } + + wchar_t **member = config_spec_get_member(spec, &config->config); + if (*member == NULL) { + *value = NULL; + return 0; + } + + *value = wstr_to_utf8(config, *member); + if (*value == NULL) { + return -1; + } + return 0; +} + + +int +PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) +{ + const PyConfigSpec *spec = initconfig_prepare_get(config, name); + if (spec == NULL) { + return -1; + } + + if (spec->type != PyConfig_MEMBER_WSTR_LIST) { + initconfig_set_error(config, "config option type is not string list"); + return -1; + } + + PyWideStringList *list = config_spec_get_member(spec, &config->config); + + *items = malloc(list->length * sizeof(char*)); + if (*items == NULL) { + config->status = _PyStatus_NO_MEMORY(); + return -1; + } + + for (Py_ssize_t i=0; i < list->length; i++) { + (*items)[i] = wstr_to_utf8(config, list->items[i]); + if ((*items)[i] == NULL) { + free(*items); + return -1; + } + } + return 0; +} + + +void +PyInitConfig_FreeStrList(size_t length, char **items) +{ + for (size_t i=0; i < length; i++) { + free(items[i]); + } + free(items); +} + + +static const PyConfigSpec* +initconfig_prepare_set(PyInitConfig *config, const char *name, int preconfig) +{ + const PyConfigSpec *spec = config_get_spec(name); + if (spec == NULL) { + initconfig_set_error(config, "unknown config option name"); + return NULL; + } + + if (preconfig) { + config->status = _Py_PreInitializeFromConfig(&config->config, NULL); + if (_PyStatus_EXCEPTION(config->status)) { + return NULL; + } + } + + return spec; +} + + +int +PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) +{ + const PyConfigSpec *spec = initconfig_prepare_set(config, name, 0); + if (spec == NULL) { + return -1; + } + + if (spec->type == PyConfig_MEMBER_INT) { + if (value < (int64_t)INT_MIN || INT_MAX < (int64_t)value) { + initconfig_set_error(config, + "config option value is out of int range"); + return -1; + } + int int_value = (int)value; + + int *member = config_spec_get_member(spec, &config->config); + *member = int_value; + } + else if (spec->type == PyConfig_MEMBER_UINT || spec->type == PyConfig_MEMBER_BOOL) { + if (value < 0 || UINT_MAX < (int64_t)value) { + initconfig_set_error(config, + "config option value is out of unsigned int range"); + return -1; + } + int int_value = (int)value; + + int *member = config_spec_get_member(spec, &config->config); + *member = int_value; + } + else if (spec->type == PyConfig_MEMBER_ULONG) { + if (value < 0 || (uint64_t)ULONG_MAX < (uint64_t)value) { + initconfig_set_error(config, + "config option value is out of unsigned long range"); + return -1; + } + unsigned long ulong_value = (unsigned long)value; + + unsigned long *member = config_spec_get_member(spec, &config->config); + *member = ulong_value; + } + else { + initconfig_set_error(config, "config option type is not int"); + return -1; + } + + if (strcmp(name, "hash_seed")) { + config->config.use_hash_seed = 1; + } + + return 0; +} + + +static wchar_t* +utf8_to_wstr(PyInitConfig *config, const char *str) +{ + wchar_t *wstr; + size_t wlen; + int res = _Py_DecodeUTF8Ex(str, strlen(str), &wstr, &wlen, NULL, _Py_ERROR_STRICT); + if (res == -2) { + initconfig_set_error(config, "decoding error"); + return NULL; + } + if (res < 0) { + config->status = _PyStatus_NO_MEMORY(); + return NULL; + } + + // Copy to use the malloc() memory allocator + size_t size = (wlen + 1) * sizeof(wchar_t); + wchar_t *wstr2 = malloc(size); + if (wstr2 == NULL) { + PyMem_RawFree(wstr); + config->status = _PyStatus_NO_MEMORY(); + return NULL; + } + + memcpy(wstr2, wstr, size); + PyMem_RawFree(wstr); + return wstr2; +} + + +int +PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char* value) +{ + const PyConfigSpec *spec = initconfig_prepare_set(config, name, 1); + if (spec == NULL) { + return -1; + } + + if (spec->type != PyConfig_MEMBER_WSTR + && spec->type != PyConfig_MEMBER_WSTR_OPT) { + initconfig_set_error(config, "config option type is not string"); + return -1; + } + + if (value == NULL && spec->type != PyConfig_MEMBER_WSTR_OPT) { + initconfig_set_error(config, "config option string cannot be NULL"); + } + + wchar_t **member = config_spec_get_member(spec, &config->config); + + *member = utf8_to_wstr(config, value); + if (*member == NULL) { + return -1; + } + return 0; +} + + +static int +_PyWideStringList_FromUTF8(PyInitConfig *config, PyWideStringList *list, + Py_ssize_t length, char * const *items) +{ + PyWideStringList wlist = _PyWideStringList_INIT; + size_t size = sizeof(wchar_t*) * length; + wlist.items = (wchar_t **)PyMem_RawMalloc(size); + if (wlist.items == NULL) { + config->status = _PyStatus_NO_MEMORY(); + return -1; + } + + for (Py_ssize_t i = 0; i < length; i++) { + wchar_t *arg = utf8_to_wstr(config, items[i]); + if (arg == NULL) { + _PyWideStringList_Clear(&wlist); + return -1; + } + wlist.items[i] = arg; + wlist.length++; + } + + _PyWideStringList_Clear(list); + *list = wlist; + return 0; +} + + +int +PyInitConfig_SetStrList(PyInitConfig *config, const char *name, + size_t length, char * const *items) +{ + const PyConfigSpec *spec = initconfig_prepare_set(config, name, 1); + if (spec == NULL) { + return -1; + } + + if (spec->type != PyConfig_MEMBER_WSTR_LIST) { + initconfig_set_error(config, "config option type is not strings list"); + return -1; + } + PyWideStringList *list = config_spec_get_member(spec, &config->config); + return _PyWideStringList_FromUTF8(config, list, length, items); +} + + +int +Py_InitializeFromInitConfig(PyInitConfig *config) +{ + config->status = Py_InitializeFromConfig(&config->config); + if (_PyStatus_EXCEPTION(config->status)) { + return -1; + } + return 0; +} + + +int +PyInitConfig_GetError(PyInitConfig* config, const char **perr_msg) +{ + if (_PyStatus_IS_EXIT(config->status)) { + char buffer[22]; // len("exit code -2147483648\0") + PyOS_snprintf(buffer, sizeof(buffer), + "exit code %i", + config->status.exitcode); + + if (config->err_msg != NULL) { + free(config->err_msg); + } + config->err_msg = strdup(buffer); + if (config->err_msg != NULL) { + *perr_msg = config->err_msg; + return 1; + } + config->status = _PyStatus_NO_MEMORY(); + } + + if (_PyStatus_IS_ERROR(config->status) && config->status.err_msg != NULL) { + *perr_msg = config->status.err_msg; + return 1; + } + else { + *perr_msg = NULL; + return 0; + } +} + + +int +PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) +{ + if (_PyStatus_IS_EXIT(config->status)) { + *exitcode = config->status.exitcode; + return 1; + } + else { + return 0; + } +} diff --git a/Python/preconfig.c b/Python/preconfig.c index 5b26c75de8b3a0..ebcfa28323f74b 100644 --- a/Python/preconfig.c +++ b/Python/preconfig.c @@ -74,33 +74,47 @@ _Py_COMP_DIAG_POP /* --- _PyArgv ---------------------------------------------------- */ -/* Decode bytes_argv using Py_DecodeLocale() */ PyStatus -_PyArgv_AsWstrList(const _PyArgv *args, PyWideStringList *list) +_PyWideStringList_FromBytes(PyWideStringList *list, + Py_ssize_t length, char * const *items) { PyWideStringList wargv = _PyWideStringList_INIT; - if (args->use_bytes_argv) { - size_t size = sizeof(wchar_t*) * args->argc; - wargv.items = (wchar_t **)PyMem_RawMalloc(size); - if (wargv.items == NULL) { - return _PyStatus_NO_MEMORY(); - } + size_t size = sizeof(wchar_t*) * length; + wargv.items = (wchar_t **)PyMem_RawMalloc(size); + if (wargv.items == NULL) { + return _PyStatus_NO_MEMORY(); + } - for (Py_ssize_t i = 0; i < args->argc; i++) { - size_t len; - wchar_t *arg = Py_DecodeLocale(args->bytes_argv[i], &len); - if (arg == NULL) { - _PyWideStringList_Clear(&wargv); - return DECODE_LOCALE_ERR("command line arguments", len); - } - wargv.items[i] = arg; - wargv.length++; + for (Py_ssize_t i = 0; i < length; i++) { + size_t len; + wchar_t *arg = Py_DecodeLocale(items[i], &len); + if (arg == NULL) { + _PyWideStringList_Clear(&wargv); + return DECODE_LOCALE_ERR("command line arguments", len); } + wargv.items[i] = arg; + wargv.length++; + } - _PyWideStringList_Clear(list); - *list = wargv; + _PyWideStringList_Clear(list); + *list = wargv; + return _PyStatus_OK(); +} + +/* Decode bytes_argv using Py_DecodeLocale() */ +PyStatus +_PyArgv_AsWstrList(const _PyArgv *args, PyWideStringList *list) +{ + if (args->use_bytes_argv) { + PyStatus status; + status = _PyWideStringList_FromBytes(list, + args->argc, args->bytes_argv); + if (_PyStatus_EXCEPTION(status)) { + return status; + } } else { + PyWideStringList wargv = _PyWideStringList_INIT; wargv.length = args->argc; wargv.items = (wchar_t **)args->wchar_argv; if (_PyWideStringList_Copy(list, &wargv) < 0) { From 79741e5ab783845522cc8c9f700544bc899c8840 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Fri, 30 Aug 2024 15:56:48 +0200 Subject: [PATCH 02/18] Fix compilation on clang --- Doc/c-api/config.rst | 2 +- Programs/_testembed.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/config.rst b/Doc/c-api/config.rst index b795dc37e7024f..2243e9e1704e46 100644 --- a/Doc/c-api/config.rst +++ b/Doc/c-api/config.rst @@ -216,9 +216,9 @@ return ``-1`` on error: PyInitConfig_Free(config); return 0; - error: // Display the error message const char *err_msg; + error: (void)PyInitConfig_GetError(config, &err_msg); printf("PYTHON INIT ERROR: %s\n", err_msg); PyInitConfig_Free(config); diff --git a/Programs/_testembed.c b/Programs/_testembed.c index b168c12f26b6b1..774dc828c95463 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -1850,8 +1850,8 @@ static int test_initconfig_api(void) Py_Finalize(); return 0; -error: const char *err_msg; +error: (void)PyInitConfig_GetError(config, &err_msg); printf("Python init failed: %s\n", err_msg); exit(1); From e04d667d622ab20800077d5d399890576573bc49 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Fri, 30 Aug 2024 16:42:12 +0200 Subject: [PATCH 03/18] Add tests on the Get API --- Include/cpython/initconfig.h | 6 +++++ Lib/test/test_embed.py | 3 +++ Programs/_testembed.c | 43 ++++++++++++++++++++++++++++++++++++ Python/initconfig.c | 1 + 4 files changed, 53 insertions(+) diff --git a/Include/cpython/initconfig.h b/Include/cpython/initconfig.h index 035a0dc92fa94c..60af7110079efb 100644 --- a/Include/cpython/initconfig.h +++ b/Include/cpython/initconfig.h @@ -281,6 +281,12 @@ PyAPI_FUNC(PyInitConfig*) PyInitConfig_Create(void); // Free memory of a initialization configuration. PyAPI_FUNC(void) PyInitConfig_Free(PyInitConfig *config); +PyAPI_FUNC(int) PyInitConfig_HasOption(PyInitConfig *config, const char *name); +PyAPI_FUNC(int) PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value); +PyAPI_FUNC(int) PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value); +PyAPI_FUNC(int) PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items); +PyAPI_FUNC(void) PyInitConfig_FreeStrList(size_t length, char **items); + // Set an integer configuration option. // Return 0 on success, or return -1 on error. PyAPI_FUNC(int) PyInitConfig_SetInt( diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 4bcbe76bdbbaf6..3076f8feebfb3b 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -1770,6 +1770,9 @@ def test_initconfig_api(self): self.check_all_configs("test_initconfig_api", config, preconfig, api=API_ISOLATED) + def test_initconfig_get_api(self): + self.run_embedded_interpreter("test_initconfig_get_api") + def test_initconfig_exit(self): self.run_embedded_interpreter("test_initconfig_exit") diff --git a/Programs/_testembed.c b/Programs/_testembed.c index 774dc828c95463..a5984323707f1e 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -1858,6 +1858,48 @@ static int test_initconfig_api(void) } +static int test_initconfig_get_api(void) +{ + PyInitConfig *config = PyInitConfig_Create(); + if (config == NULL) { + printf("Init allocation error\n"); + return 1; + } + + // test PyInitConfig_HasOption() + assert(PyInitConfig_HasOption(config, "verbose") == 1); + assert(PyInitConfig_HasOption(config, "non-existent") == 0); + + // test PyInitConfig_GetInt() + int64_t value; + assert(PyInitConfig_GetInt(config, "dev_mode", &value) == 0); + assert(value == 0); + assert(PyInitConfig_SetInt(config, "dev_mode", 1) == 0); + assert(PyInitConfig_GetInt(config, "dev_mode", &value) == 0); + assert(value == 1); + + // test PyInitConfig_GetStr() + char *str; + assert(PyInitConfig_SetStr(config, "program_name", PROGRAM_NAME_UTF8) == 0); + assert(PyInitConfig_GetStr(config, "program_name", &str) == 0); + assert(strcmp(str, PROGRAM_NAME_UTF8) == 0); + free(str); + + // test PyInitConfig_GetStrList() and PyInitConfig_FreeStrList() + char* xoptions[] = {"faulthandler"}; + assert(PyInitConfig_SetStrList(config, "xoptions", + Py_ARRAY_LENGTH(xoptions), xoptions) == 0); + size_t length; + char **items; + assert(PyInitConfig_GetStrList(config, "xoptions", &length, &items) == 0); + assert(length == 1); + assert(strcmp(items[0], "faulthandler") == 0); + PyInitConfig_FreeStrList(length, items); + + return 0; +} + + static int test_initconfig_exit(void) { PyInitConfig *config = PyInitConfig_Create(); @@ -2306,6 +2348,7 @@ static struct TestCase TestCases[] = { {"test_init_warnoptions", test_init_warnoptions}, {"test_init_set_config", test_init_set_config}, {"test_initconfig_api", test_initconfig_api}, + {"test_initconfig_get_api", test_initconfig_get_api}, {"test_initconfig_exit", test_initconfig_exit}, {"test_run_main", test_run_main}, {"test_run_main_loop", test_run_main_loop}, diff --git a/Python/initconfig.c b/Python/initconfig.c index 6e0ae7b0fbc2a5..f8bed48d0a2259 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3462,6 +3462,7 @@ PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, } PyWideStringList *list = config_spec_get_member(spec, &config->config); + *length = list->length; *items = malloc(list->length * sizeof(char*)); if (*items == NULL) { From 6ca4b41a8ac021e9a37ed13a459949c3b12f2991 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Fri, 30 Aug 2024 16:46:00 +0200 Subject: [PATCH 04/18] Remove preconfiguration; add raw_member --- Python/initconfig.c | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/Python/initconfig.c b/Python/initconfig.c index f8bed48d0a2259..7cb7c18547bb56 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3277,7 +3277,6 @@ _Py_DumpPathConfig(PyThreadState *tstate) // --- PyInitConfig API --------------------------------------------------- struct PyInitConfig { - // FIXME: add preconfig PyConfig config; PyStatus status; char *err_msg; @@ -3492,21 +3491,15 @@ PyInitConfig_FreeStrList(size_t length, char **items) static const PyConfigSpec* -initconfig_prepare_set(PyInitConfig *config, const char *name, int preconfig) +initconfig_prepare_set(PyInitConfig *config, const char *name, + void **raw_member) { const PyConfigSpec *spec = config_get_spec(name); if (spec == NULL) { initconfig_set_error(config, "unknown config option name"); return NULL; } - - if (preconfig) { - config->status = _Py_PreInitializeFromConfig(&config->config, NULL); - if (_PyStatus_EXCEPTION(config->status)) { - return NULL; - } - } - + *raw_member = config_spec_get_member(spec, &config->config); return spec; } @@ -3514,7 +3507,8 @@ initconfig_prepare_set(PyInitConfig *config, const char *name, int preconfig) int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) { - const PyConfigSpec *spec = initconfig_prepare_set(config, name, 0); + void *raw_member; + const PyConfigSpec *spec = initconfig_prepare_set(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3527,7 +3521,7 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) } int int_value = (int)value; - int *member = config_spec_get_member(spec, &config->config); + int *member = raw_member; *member = int_value; } else if (spec->type == PyConfig_MEMBER_UINT || spec->type == PyConfig_MEMBER_BOOL) { @@ -3538,7 +3532,7 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) } int int_value = (int)value; - int *member = config_spec_get_member(spec, &config->config); + int *member = raw_member; *member = int_value; } else if (spec->type == PyConfig_MEMBER_ULONG) { @@ -3549,7 +3543,7 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) } unsigned long ulong_value = (unsigned long)value; - unsigned long *member = config_spec_get_member(spec, &config->config); + unsigned long *member = raw_member; *member = ulong_value; } else { @@ -3598,7 +3592,8 @@ utf8_to_wstr(PyInitConfig *config, const char *str) int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char* value) { - const PyConfigSpec *spec = initconfig_prepare_set(config, name, 1); + void *raw_member; + const PyConfigSpec *spec = initconfig_prepare_set(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3613,7 +3608,7 @@ PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char* value) initconfig_set_error(config, "config option string cannot be NULL"); } - wchar_t **member = config_spec_get_member(spec, &config->config); + wchar_t **member = raw_member; *member = utf8_to_wstr(config, value); if (*member == NULL) { @@ -3655,7 +3650,8 @@ int PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items) { - const PyConfigSpec *spec = initconfig_prepare_set(config, name, 1); + void *raw_member; + const PyConfigSpec *spec = initconfig_prepare_set(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3664,7 +3660,7 @@ PyInitConfig_SetStrList(PyInitConfig *config, const char *name, initconfig_set_error(config, "config option type is not strings list"); return -1; } - PyWideStringList *list = config_spec_get_member(spec, &config->config); + PyWideStringList *list = raw_member; return _PyWideStringList_FromUTF8(config, list, length, items); } @@ -3676,6 +3672,7 @@ Py_InitializeFromInitConfig(PyInitConfig *config) if (_PyStatus_EXCEPTION(config->status)) { return -1; } + return 0; } From 693c72b689807cecedb5520543f3ab3e2880a58d Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Mon, 2 Sep 2024 23:38:35 +0200 Subject: [PATCH 05/18] Rename config_get_spec() to initconfig_get_spec() --- Python/initconfig.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Python/initconfig.c b/Python/initconfig.c index 2417cf0db67074..ce5d358591963b 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3456,7 +3456,7 @@ initconfig_set_error(PyInitConfig *config, const char *err_msg) static const PyConfigSpec* -config_get_spec(const char *name) +initconfig_get_spec(const char *name) { const PyConfigSpec *spec = PYCONFIG_SPEC; for (; spec->name != NULL; spec++) { @@ -3471,7 +3471,7 @@ config_get_spec(const char *name) int PyInitConfig_HasOption(PyInitConfig *config, const char *name) { - const PyConfigSpec *spec = config_get_spec(name); + const PyConfigSpec *spec = initconfig_get_spec(name); return (spec != NULL); } @@ -3479,7 +3479,7 @@ PyInitConfig_HasOption(PyInitConfig *config, const char *name) static const PyConfigSpec* initconfig_prepare_get(PyInitConfig *config, const char *name) { - const PyConfigSpec *spec = config_get_spec(name); + const PyConfigSpec *spec = initconfig_get_spec(name); if (spec == NULL) { initconfig_set_error(config, "unknown config option name"); return NULL; @@ -3632,7 +3632,7 @@ static const PyConfigSpec* initconfig_prepare_set(PyInitConfig *config, const char *name, void **raw_member) { - const PyConfigSpec *spec = config_get_spec(name); + const PyConfigSpec *spec = initconfig_get_spec(name); if (spec == NULL) { initconfig_set_error(config, "unknown config option name"); return NULL; From fcc7d09cb9c027487f23e96daa7b7402516c9cbd Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Mon, 2 Sep 2024 23:51:10 +0200 Subject: [PATCH 06/18] Add PyInitConfig.preconfig --- Lib/test/test_embed.py | 10 ++-- Programs/_testembed.c | 11 +++++ Python/initconfig.c | 105 +++++++++++++++++++++++------------------ 3 files changed, 76 insertions(+), 50 deletions(-) diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 7165165b521bfc..17656d7b3d7fb4 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -1756,17 +1756,17 @@ def test_init_set_config(self): api=API_ISOLATED) def test_initconfig_api(self): - preconfig = {} + preconfig = { + 'configure_locale': True, + } config = { 'pycache_prefix': 'conf_pycache_prefix', 'xoptions': {'faulthandler': True}, 'hash_seed': 10, 'use_hash_seed': True, } - preconfig['allocator'] = PYMEM_ALLOCATOR_DEBUG - preconfig['dev_mode'] = 1 - config['dev_mode'] = 1 - config['warnoptions'] = ['default'] + config_dev_mode(preconfig, config) + config['faulthandler'] = 0 self.check_all_configs("test_initconfig_api", config, preconfig, api=API_ISOLATED) diff --git a/Programs/_testembed.c b/Programs/_testembed.c index a5984323707f1e..8a93633339244a 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -1814,6 +1814,10 @@ static int test_initconfig_api(void) return 1; } + if (PyInitConfig_SetInt(config, "configure_locale", 1) < 0) { + goto error; + } + if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) { goto error; } @@ -1878,6 +1882,13 @@ static int test_initconfig_get_api(void) assert(PyInitConfig_GetInt(config, "dev_mode", &value) == 0); assert(value == 1); + // test PyInitConfig_GetInt() on a PyPreConfig option + assert(PyInitConfig_GetInt(config, "utf8_mode", &value) == 0); + assert(value == 0); + assert(PyInitConfig_SetInt(config, "utf8_mode", 1) == 0); + assert(PyInitConfig_GetInt(config, "utf8_mode", &value) == 0); + assert(value == 1); + // test PyInitConfig_GetStr() char *str; assert(PyInitConfig_SetStr(config, "program_name", PROGRAM_NAME_UTF8) == 0); diff --git a/Python/initconfig.c b/Python/initconfig.c index ce5d358591963b..affb26f982999b 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -1121,6 +1121,13 @@ config_spec_get_member(const PyConfigSpec *spec, const PyConfig *config) } +static inline void* +preconfig_spec_get_member(const PyConfigSpec *spec, const PyPreConfig *preconfig) +{ + return (char *)preconfig + spec->offset; +} + + PyStatus _PyConfig_Copy(PyConfig *config, const PyConfig *config2) { @@ -3415,6 +3422,7 @@ _Py_DumpPathConfig(PyThreadState *tstate) // --- PyInitConfig API --------------------------------------------------- struct PyInitConfig { + PyPreConfig preconfig; PyConfig config; PyStatus status; char *err_msg; @@ -3434,6 +3442,7 @@ PyInitConfig_Create(void) if (config == NULL) { return NULL; } + PyPreConfig_InitIsolatedConfig(&config->preconfig); PyConfig_InitIsolatedConfig(&config->config); config->status = _PyStatus_OK(); return config; @@ -3456,9 +3465,8 @@ initconfig_set_error(PyInitConfig *config, const char *err_msg) static const PyConfigSpec* -initconfig_get_spec(const char *name) +initconfig_find_spec(const PyConfigSpec *spec, const char *name) { - const PyConfigSpec *spec = PYCONFIG_SPEC; for (; spec->name != NULL; spec++) { if (strcmp(name, spec->name) == 0) { return spec; @@ -3471,27 +3479,36 @@ initconfig_get_spec(const char *name) int PyInitConfig_HasOption(PyInitConfig *config, const char *name) { - const PyConfigSpec *spec = initconfig_get_spec(name); + const PyConfigSpec *spec = initconfig_find_spec(PYCONFIG_SPEC, name); return (spec != NULL); } static const PyConfigSpec* -initconfig_prepare_get(PyInitConfig *config, const char *name) +initconfig_prepare(PyInitConfig *config, const char *name, void **raw_member) { - const PyConfigSpec *spec = initconfig_get_spec(name); - if (spec == NULL) { - initconfig_set_error(config, "unknown config option name"); - return NULL; + const PyConfigSpec *spec = initconfig_find_spec(PYCONFIG_SPEC, name); + if (spec != NULL) { + *raw_member = config_spec_get_member(spec, &config->config); + return spec; } - return spec; + + spec = initconfig_find_spec(PYPRECONFIG_SPEC, name); + if (spec != NULL) { + *raw_member = preconfig_spec_get_member(spec, &config->preconfig); + return spec; + } + + initconfig_set_error(config, "unknown config option name"); + return NULL; } int PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) { - const PyConfigSpec *spec = initconfig_prepare_get(config, name); + void *raw_member; + const PyConfigSpec *spec = initconfig_prepare(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3501,14 +3518,14 @@ PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) case PyConfig_MEMBER_UINT: case PyConfig_MEMBER_BOOL: { - int *member = config_spec_get_member(spec, &config->config); + int *member = raw_member; *value = *member; break; } case PyConfig_MEMBER_ULONG: { - unsigned long *member = config_spec_get_member(spec, &config->config); + unsigned long *member = raw_member; #if SIZEOF_LONG >= 8 if ((unsigned long)INT64_MAX < *member) { initconfig_set_error(config, @@ -3560,7 +3577,8 @@ wstr_to_utf8(PyInitConfig *config, wchar_t *wstr) int PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) { - const PyConfigSpec *spec = initconfig_prepare_get(config, name); + void *raw_member; + const PyConfigSpec *spec = initconfig_prepare(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3571,7 +3589,7 @@ PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) return -1; } - wchar_t **member = config_spec_get_member(spec, &config->config); + wchar_t **member = raw_member; if (*member == NULL) { *value = NULL; return 0; @@ -3588,7 +3606,8 @@ PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) { - const PyConfigSpec *spec = initconfig_prepare_get(config, name); + void *raw_member; + const PyConfigSpec *spec = initconfig_prepare(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3598,7 +3617,7 @@ PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, return -1; } - PyWideStringList *list = config_spec_get_member(spec, &config->config); + PyWideStringList *list = raw_member; *length = list->length; *items = malloc(list->length * sizeof(char*)); @@ -3628,25 +3647,11 @@ PyInitConfig_FreeStrList(size_t length, char **items) } -static const PyConfigSpec* -initconfig_prepare_set(PyInitConfig *config, const char *name, - void **raw_member) -{ - const PyConfigSpec *spec = initconfig_get_spec(name); - if (spec == NULL) { - initconfig_set_error(config, "unknown config option name"); - return NULL; - } - *raw_member = config_spec_get_member(spec, &config->config); - return spec; -} - - int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) { void *raw_member; - const PyConfigSpec *spec = initconfig_prepare_set(config, name, &raw_member); + const PyConfigSpec *spec = initconfig_prepare(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3731,7 +3736,7 @@ int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char* value) { void *raw_member; - const PyConfigSpec *spec = initconfig_prepare_set(config, name, &raw_member); + const PyConfigSpec *spec = initconfig_prepare(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3789,7 +3794,7 @@ PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items) { void *raw_member; - const PyConfigSpec *spec = initconfig_prepare_set(config, name, &raw_member); + const PyConfigSpec *spec = initconfig_prepare(config, name, &raw_member); if (spec == NULL) { return -1; } @@ -3803,18 +3808,6 @@ PyInitConfig_SetStrList(PyInitConfig *config, const char *name, } -int -Py_InitializeFromInitConfig(PyInitConfig *config) -{ - config->status = Py_InitializeFromConfig(&config->config); - if (_PyStatus_EXCEPTION(config->status)) { - return -1; - } - - return 0; -} - - int PyInitConfig_GetError(PyInitConfig* config, const char **perr_msg) { @@ -3859,6 +3852,28 @@ PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) } +int +Py_InitializeFromInitConfig(PyInitConfig *config) +{ + _PyPreConfig_GetConfig(&config->preconfig, &config->config); + + config->status = Py_PreInitializeFromArgs( + &config->preconfig, + config->config.argv.length, + config->config.argv.items); + if (_PyStatus_EXCEPTION(config->status)) { + return -1; + } + + config->status = Py_InitializeFromConfig(&config->config); + if (_PyStatus_EXCEPTION(config->status)) { + return -1; + } + + return 0; +} + + // --- PyConfig_Get() ------------------------------------------------------- static const PyConfigSpec* From 96c4b08754b620f55248563931978e5486a493f4 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 00:33:52 +0200 Subject: [PATCH 07/18] Cleanup header file Move also PyInitConfig_GetError() and PyInitConfig_GetExitCode() implementations. --- Include/cpython/initconfig.h | 38 ++-------------- Python/initconfig.c | 88 ++++++++++++++++++------------------ 2 files changed, 47 insertions(+), 79 deletions(-) diff --git a/Include/cpython/initconfig.h b/Include/cpython/initconfig.h index 59ada8b6377989..053b533cee74ee 100644 --- a/Include/cpython/initconfig.h +++ b/Include/cpython/initconfig.h @@ -280,66 +280,34 @@ PyAPI_FUNC(void) Py_GetArgcArgv(int *argc, wchar_t ***argv); typedef struct PyInitConfig PyInitConfig; -// Create a new initialization configuration using the Isolated configuration -// defaults. -// It must be freed by PyInitConfig_Free(). -// Return NULL on memory allocation failure. PyAPI_FUNC(PyInitConfig*) PyInitConfig_Create(void); - -// Free memory of a initialization configuration. PyAPI_FUNC(void) PyInitConfig_Free(PyInitConfig *config); +PyAPI_FUNC(int) PyInitConfig_GetError(PyInitConfig* config, const char **err_msg); +PyAPI_FUNC(int) PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode); + PyAPI_FUNC(int) PyInitConfig_HasOption(PyInitConfig *config, const char *name); PyAPI_FUNC(int) PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value); PyAPI_FUNC(int) PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value); PyAPI_FUNC(int) PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items); PyAPI_FUNC(void) PyInitConfig_FreeStrList(size_t length, char **items); -// Set an integer configuration option. -// Return 0 on success, or return -1 on error. PyAPI_FUNC(int) PyInitConfig_SetInt( PyInitConfig *config, const char *name, int64_t value); - -// Set a string configuration option from a bytes string. -// -// The bytes string is decoded by Py_DecodeLocale(). Preinitialize Python if -// needed to ensure that encodings are properly configured. -// -// Return 0 on success, or return -1 on error. PyAPI_FUNC(int) PyInitConfig_SetStr( PyInitConfig *config, const char *name, const char *value); - -// Set a string list configuration option from bytes strings. -// -// The bytes strings are decoded by Py_DecodeLocale(). Preinitialize Python if -// needed to ensure that encodings are properly configured. -// -// Return 0 on success, or return -1 on error. PyAPI_FUNC(int) PyInitConfig_SetStrList( PyInitConfig *config, const char *name, size_t length, char * const *items); -// Initialize Python from the initialization configuration. -// Return 0 on success. -// Return -1 if Python wants to exit and on error PyAPI_FUNC(int) Py_InitializeFromInitConfig(PyInitConfig *config); -// Get the error message. -// Set *err_msg and return 1 if an error is set. -// Set *err_msg to NULL and return 0 otherwise. -PyAPI_FUNC(int) PyInitConfig_GetError(PyInitConfig* config, const char **err_msg); - -// Get the exit code. -// Set '*exitcode' and return 1 if an exit code is set. -// Return 0 otherwise. -PyAPI_FUNC(int) PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode); - #ifdef __cplusplus } diff --git a/Python/initconfig.c b/Python/initconfig.c index affb26f982999b..f16d96937a0a33 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3457,6 +3457,50 @@ PyInitConfig_Free(PyInitConfig *config) } +int +PyInitConfig_GetError(PyInitConfig* config, const char **perr_msg) +{ + if (_PyStatus_IS_EXIT(config->status)) { + char buffer[22]; // len("exit code -2147483648\0") + PyOS_snprintf(buffer, sizeof(buffer), + "exit code %i", + config->status.exitcode); + + if (config->err_msg != NULL) { + free(config->err_msg); + } + config->err_msg = strdup(buffer); + if (config->err_msg != NULL) { + *perr_msg = config->err_msg; + return 1; + } + config->status = _PyStatus_NO_MEMORY(); + } + + if (_PyStatus_IS_ERROR(config->status) && config->status.err_msg != NULL) { + *perr_msg = config->status.err_msg; + return 1; + } + else { + *perr_msg = NULL; + return 0; + } +} + + +int +PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) +{ + if (_PyStatus_IS_EXIT(config->status)) { + *exitcode = config->status.exitcode; + return 1; + } + else { + return 0; + } +} + + static void initconfig_set_error(PyInitConfig *config, const char *err_msg) { @@ -3808,50 +3852,6 @@ PyInitConfig_SetStrList(PyInitConfig *config, const char *name, } -int -PyInitConfig_GetError(PyInitConfig* config, const char **perr_msg) -{ - if (_PyStatus_IS_EXIT(config->status)) { - char buffer[22]; // len("exit code -2147483648\0") - PyOS_snprintf(buffer, sizeof(buffer), - "exit code %i", - config->status.exitcode); - - if (config->err_msg != NULL) { - free(config->err_msg); - } - config->err_msg = strdup(buffer); - if (config->err_msg != NULL) { - *perr_msg = config->err_msg; - return 1; - } - config->status = _PyStatus_NO_MEMORY(); - } - - if (_PyStatus_IS_ERROR(config->status) && config->status.err_msg != NULL) { - *perr_msg = config->status.err_msg; - return 1; - } - else { - *perr_msg = NULL; - return 0; - } -} - - -int -PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) -{ - if (_PyStatus_IS_EXIT(config->status)) { - *exitcode = config->status.exitcode; - return 1; - } - else { - return 0; - } -} - - int Py_InitializeFromInitConfig(PyInitConfig *config) { From de3c5f360a4d9274ccb92068ec054381f17fedf8 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 00:40:21 +0200 Subject: [PATCH 08/18] Move documentation into init_config.rst --- Doc/c-api/config.rst | 227 ----------------------------------- Doc/c-api/index.rst | 1 - Doc/c-api/init.rst | 3 +- Doc/c-api/init_config.rst | 244 ++++++++++++++++++++++++++++++++++++-- 4 files changed, 233 insertions(+), 242 deletions(-) delete mode 100644 Doc/c-api/config.rst diff --git a/Doc/c-api/config.rst b/Doc/c-api/config.rst deleted file mode 100644 index 2243e9e1704e46..00000000000000 --- a/Doc/c-api/config.rst +++ /dev/null @@ -1,227 +0,0 @@ -.. highlight:: c - -.. _config-c-api: - -******************** -Python Configuration -******************** - -.. versionadded:: 3.14 - -C API to configure the Python initialization (:pep:`741`). - -See also :ref:`Python Initialization Configuration `. - - -Initialize Python -================= - -Create Config -------------- - -.. c:struct:: PyInitConfig - - Opaque structure to configure the Python initialization. - - -.. c:function:: PyInitConfig* PyInitConfig_Create(void) - - Create a new initialization configuration using :ref:`Isolated Configuration - ` default values. - - It must be freed by :c:func:`PyInitConfig_Free`. - - Return ``NULL`` on memory allocation failure. - - -.. c:function:: void PyInitConfig_Free(PyInitConfig *config) - - Free memory of the initialization configuration *config*. - - -Get Options ------------ - -The configuration option *name* parameter must be a non-NULL -null-terminated UTF-8 encoded string. - -.. c:function:: int PyInitConfig_HasOption(PyInitConfig *config, const char *name) - - Test if the configuration has an option called *name*. - - Return ``1`` if the option exists, or return ``0`` otherwise. - - -.. c:function:: int PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) - - Get an integer configuration option. - - * Set *\*value*, and return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -.. c:function:: int PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) - - Get a string configuration option as a null-terminated UTF-8 - encoded string. - - * Set *\*value*, and return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - On success, the string must be released with ``free(value)``. - - -.. c:function:: int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) - - Get a string list configuration option as an array of - null-terminated UTF-8 encoded strings. - - * Set *\*length* and *\*value*, and return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - On success, the string list must be released with - ``PyInitConfig_FreeStrList(length, items)``. - - -.. c:function:: void PyInitConfig_FreeStrList(size_t length, char **items) - - Free memory of a string list created by - ``PyInitConfig_GetStrList()``. - - -Set Options ------------ - -The configuration option *name* parameter must be a non-NULL null-terminated -UTF-8 encoded string. - -Some configuration options have side effects on other options. This logic is -only implemented when ``Py_InitializeFromInitConfig()`` is called, not by the -"Set" functions below. For example, setting ``dev_mode`` to ``1`` does not set -``faulthandler`` to ``1``. - -.. c:function:: int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) - - Set an integer configuration option. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -.. c:function:: int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char *value) - - Set a string configuration option from a null-terminated UTF-8 - encoded string. The string is copied. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -.. c:function:: int PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items) - - Set a string list configuration option from an array of - null-terminated UTF-8 encoded strings. The string list is copied. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -Initialize Python ------------------ - -.. c:function:: int Py_InitializeFromInitConfig(PyInitConfig *config) - - Initialize Python from the initialization configuration. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - * Set an exit code in *config* and return ``-1`` if Python wants to - exit. - - See ``PyInitConfig_GetExitcode()`` for the exit code case. - - -Error Handling --------------- - -.. c:function:: int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg) - - Get the *config* error message. - - * Set *\*err_msg* and return ``1`` if an error is set. - * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise. - - An error message is an UTF-8 encoded string. - - If *config* has an exit code, format the exit code as an error - message. - - The error message remains valid until another ``PyInitConfig`` - function is called with *config*. The caller doesn't have to free the - error message. - - -.. c:function:: int PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) - - Get the *config* exit code. - - * Set *\*exitcode* and return ``1`` if Python wants to exit. - * Return ``0`` if *config* has no exit code set. - - Only the ``Py_InitializeFromInitConfig()`` function can set an exit - code if the ``parse_argv`` option is non-zero. - - An exit code can be set when parsing the command line failed (exit - code ``2``) or when a command line option asks to display the command - line help (exit code ``0``). - - -Example -======= - -Example initializing Python, set configuration options of various types, -return ``-1`` on error: - -.. code-block:: c - - int init_python(void) - { - PyInitConfig *config = PyInitConfig_Create(); - if (config == NULL) { - printf("PYTHON INIT ERROR: memory allocation failed\n"); - return -1; - } - - // Set an integer (dev mode) - if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) { - goto error; - } - - // Set a list of UTF-8 strings (argv) - char *argv[] = {"my_program", "-c", "pass"}; - if (PyInitConfig_SetStrList(config, "argv", - Py_ARRAY_LENGTH(argv), argv) < 0) { - goto error; - } - - // Set a UTF-8 string (program name) - if (PyInitConfig_SetStr(config, "program_name", L"my_program") < 0) { - goto error; - } - - // Initialize Python with the configuration - if (Py_InitializeFromInitConfig(config) < 0) { - goto error; - } - PyInitConfig_Free(config); - return 0; - - // Display the error message - const char *err_msg; - error: - (void)PyInitConfig_GetError(config, &err_msg); - printf("PYTHON INIT ERROR: %s\n", err_msg); - PyInitConfig_Free(config); - - return -1; - } diff --git a/Doc/c-api/index.rst b/Doc/c-api/index.rst index fa2ada706072aa..ba56b03c6ac8e7 100644 --- a/Doc/c-api/index.rst +++ b/Doc/c-api/index.rst @@ -22,7 +22,6 @@ document the API functions in detail. concrete.rst init.rst init_config.rst - config.rst memory.rst objimpl.rst apiabiversion.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 4147deff4f839d..561c8a1b879bae 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -7,8 +7,7 @@ Initialization, Finalization, and Threads ***************************************** -See also the :ref:`Python Initialization Configuration ` -and the :ref:`Python Configuration ` +See also the :ref:`Python Initialization Configuration `. .. _pre-init-safe: diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 14753b1b6c4b35..5839c9004da7c0 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -6,6 +6,9 @@ Python Initialization Configuration *********************************** +PyConfig C API +============== + .. versionadded:: 3.8 Python can be initialized with :c:func:`Py_InitializeFromConfig` and the @@ -27,15 +30,14 @@ There are two kinds of configuration: The :c:func:`Py_RunMain` function can be used to write a customized Python program. -See also :ref:`Initialization, Finalization, and Threads ` -and the :ref:`Python Configuration `. +See also :ref:`Initialization, Finalization, and Threads `. .. seealso:: :pep:`587` "Python Initialization Configuration". Example -======= +------- Example of customized Python always running in isolated mode:: @@ -74,7 +76,7 @@ Example of customized Python always running in isolated mode:: PyWideStringList -================ +---------------- .. c:type:: PyWideStringList @@ -117,7 +119,7 @@ PyWideStringList List items. PyStatus -======== +-------- .. c:type:: PyStatus @@ -211,7 +213,7 @@ Example:: PyPreConfig -=========== +----------- .. c:type:: PyPreConfig @@ -361,7 +363,7 @@ PyPreConfig .. _c-preinit: Preinitialize Python with PyPreConfig -===================================== +------------------------------------- The preinitialization of Python: @@ -441,7 +443,7 @@ the :ref:`Python UTF-8 Mode `:: PyConfig -======== +-------- .. c:type:: PyConfig @@ -1350,7 +1352,7 @@ the :option:`-X` command line option. Initialization with PyConfig -============================ +---------------------------- Function to initialize Python: @@ -1462,7 +1464,7 @@ initialization:: .. _init-isolated-conf: Isolated Configuration -====================== +---------------------- :c:func:`PyPreConfig_InitIsolatedConfig` and :c:func:`PyConfig_InitIsolatedConfig` functions create a configuration to @@ -1482,7 +1484,7 @@ to avoid computing the default path configuration. .. _init-python-config: Python Configuration -==================== +-------------------- :c:func:`PyPreConfig_InitPythonConfig` and :c:func:`PyConfig_InitPythonConfig` functions create a configuration to build a customized Python which behaves as @@ -1500,7 +1502,7 @@ and :ref:`Python UTF-8 Mode ` .. _init-path-config: Python Path Configuration -========================= +------------------------- :c:type:`PyConfig` contains multiple fields for the path configuration: @@ -1586,6 +1588,224 @@ The ``__PYVENV_LAUNCHER__`` environment variable is used to set :c:member:`PyConfig.base_executable`. +PyInitConfig C API +================== + +C API to configure the Python initialization (:pep:`741`). + +.. versionadded:: 3.14 + +Create Config +------------- + +.. c:struct:: PyInitConfig + + Opaque structure to configure the Python initialization. + + +.. c:function:: PyInitConfig* PyInitConfig_Create(void) + + Create a new initialization configuration using :ref:`Isolated Configuration + ` default values. + + It must be freed by :c:func:`PyInitConfig_Free`. + + Return ``NULL`` on memory allocation failure. + + +.. c:function:: void PyInitConfig_Free(PyInitConfig *config) + + Free memory of the initialization configuration *config*. + + +Get Options +----------- + +The configuration option *name* parameter must be a non-NULL +null-terminated UTF-8 encoded string. + +.. c:function:: int PyInitConfig_HasOption(PyInitConfig *config, const char *name) + + Test if the configuration has an option called *name*. + + Return ``1`` if the option exists, or return ``0`` otherwise. + + +.. c:function:: int PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) + + Get an integer configuration option. + + * Set *\*value*, and return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +.. c:function:: int PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) + + Get a string configuration option as a null-terminated UTF-8 + encoded string. + + * Set *\*value*, and return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + On success, the string must be released with ``free(value)``. + + +.. c:function:: int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) + + Get a string list configuration option as an array of + null-terminated UTF-8 encoded strings. + + * Set *\*length* and *\*value*, and return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + On success, the string list must be released with + ``PyInitConfig_FreeStrList(length, items)``. + + +.. c:function:: void PyInitConfig_FreeStrList(size_t length, char **items) + + Free memory of a string list created by + ``PyInitConfig_GetStrList()``. + + +Set Options +----------- + +The configuration option *name* parameter must be a non-NULL null-terminated +UTF-8 encoded string. + +Some configuration options have side effects on other options. This logic is +only implemented when ``Py_InitializeFromInitConfig()`` is called, not by the +"Set" functions below. For example, setting ``dev_mode`` to ``1`` does not set +``faulthandler`` to ``1``. + +.. c:function:: int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) + + Set an integer configuration option. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +.. c:function:: int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char *value) + + Set a string configuration option from a null-terminated UTF-8 + encoded string. The string is copied. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +.. c:function:: int PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items) + + Set a string list configuration option from an array of + null-terminated UTF-8 encoded strings. The string list is copied. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + + +Initialize Python +----------------- + +.. c:function:: int Py_InitializeFromInitConfig(PyInitConfig *config) + + Initialize Python from the initialization configuration. + + * Return ``0`` on success. + * Set an error in *config* and return ``-1`` on error. + * Set an exit code in *config* and return ``-1`` if Python wants to + exit. + + See ``PyInitConfig_GetExitcode()`` for the exit code case. + + +Error Handling +-------------- + +.. c:function:: int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg) + + Get the *config* error message. + + * Set *\*err_msg* and return ``1`` if an error is set. + * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise. + + An error message is an UTF-8 encoded string. + + If *config* has an exit code, format the exit code as an error + message. + + The error message remains valid until another ``PyInitConfig`` + function is called with *config*. The caller doesn't have to free the + error message. + + +.. c:function:: int PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) + + Get the *config* exit code. + + * Set *\*exitcode* and return ``1`` if Python wants to exit. + * Return ``0`` if *config* has no exit code set. + + Only the ``Py_InitializeFromInitConfig()`` function can set an exit + code if the ``parse_argv`` option is non-zero. + + An exit code can be set when parsing the command line failed (exit + code ``2``) or when a command line option asks to display the command + line help (exit code ``0``). + + +Example +------- + +Example initializing Python, set configuration options of various types, +return ``-1`` on error: + +.. code-block:: c + + int init_python(void) + { + PyInitConfig *config = PyInitConfig_Create(); + if (config == NULL) { + printf("PYTHON INIT ERROR: memory allocation failed\n"); + return -1; + } + + // Set an integer (dev mode) + if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) { + goto error; + } + + // Set a list of UTF-8 strings (argv) + char *argv[] = {"my_program", "-c", "pass"}; + if (PyInitConfig_SetStrList(config, "argv", + Py_ARRAY_LENGTH(argv), argv) < 0) { + goto error; + } + + // Set a UTF-8 string (program name) + if (PyInitConfig_SetStr(config, "program_name", L"my_program") < 0) { + goto error; + } + + // Initialize Python with the configuration + if (Py_InitializeFromInitConfig(config) < 0) { + goto error; + } + PyInitConfig_Free(config); + return 0; + + // Display the error message + const char *err_msg; + error: + (void)PyInitConfig_GetError(config, &err_msg); + printf("PYTHON INIT ERROR: %s\n", err_msg); + PyInitConfig_Free(config); + + return -1; + } + + Py_RunMain() ============ From 27c21b9b9b61be3105dcd0b8db227a20aebff64a Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 00:45:30 +0200 Subject: [PATCH 09/18] Document Error Handling earlier --- Doc/c-api/init_config.rst | 70 +++++++++---------- Doc/whatsnew/3.14.rst | 4 +- ...-08-30-14-02-17.gh-issue-107954.TPvj4u.rst | 4 +- 3 files changed, 39 insertions(+), 39 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 5839c9004da7c0..248a7a9e24fd88 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1618,6 +1618,41 @@ Create Config Free memory of the initialization configuration *config*. +Error Handling +-------------- + +.. c:function:: int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg) + + Get the *config* error message. + + * Set *\*err_msg* and return ``1`` if an error is set. + * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise. + + An error message is an UTF-8 encoded string. + + If *config* has an exit code, format the exit code as an error + message. + + The error message remains valid until another ``PyInitConfig`` + function is called with *config*. The caller doesn't have to free the + error message. + + +.. c:function:: int PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) + + Get the *config* exit code. + + * Set *\*exitcode* and return ``1`` if Python wants to exit. + * Return ``0`` if *config* has no exit code set. + + Only the ``Py_InitializeFromInitConfig()`` function can set an exit + code if the ``parse_argv`` option is non-zero. + + An exit code can be set when parsing the command line failed (exit + code ``2``) or when a command line option asks to display the command + line help (exit code ``0``). + + Get Options ----------- @@ -1720,41 +1755,6 @@ Initialize Python See ``PyInitConfig_GetExitcode()`` for the exit code case. -Error Handling --------------- - -.. c:function:: int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg) - - Get the *config* error message. - - * Set *\*err_msg* and return ``1`` if an error is set. - * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise. - - An error message is an UTF-8 encoded string. - - If *config* has an exit code, format the exit code as an error - message. - - The error message remains valid until another ``PyInitConfig`` - function is called with *config*. The caller doesn't have to free the - error message. - - -.. c:function:: int PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) - - Get the *config* exit code. - - * Set *\*exitcode* and return ``1`` if Python wants to exit. - * Return ``0`` if *config* has no exit code set. - - Only the ``Py_InitializeFromInitConfig()`` function can set an exit - code if the ``parse_argv`` option is non-zero. - - An exit code can be set when parsing the command line failed (exit - code ``2``) or when a command line option asks to display the command - line help (exit code ``0``). - - Example ------- diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index d54524d30142f3..f2a73ff61728f1 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -505,6 +505,8 @@ New Features * :c:func:`PyInitConfig_Create` * :c:func:`PyInitConfig_Free` + * :c:func:`PyInitConfig_GetError` + * :c:func:`PyInitConfig_GetExitCode` * :c:func:`PyInitConfig_HasOption` * :c:func:`PyInitConfig_GetInt` * :c:func:`PyInitConfig_GetStr` @@ -514,8 +516,6 @@ New Features * :c:func:`PyInitConfig_SetStr` * :c:func:`PyInitConfig_SetStrList` * :c:func:`Py_InitializeFromInitConfig` - * :c:func:`PyInitConfig_GetError` - * :c:func:`PyInitConfig_GetExitCode` (Contributed by Victor Stinner in :gh:`107954`.) diff --git a/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst b/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst index 6caff79d2b3a1c..8496b91f183d25 100644 --- a/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst +++ b/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst @@ -2,6 +2,8 @@ Add functions to configure the Python initialization: * :c:func:`PyInitConfig_Create` * :c:func:`PyInitConfig_Free` +* :c:func:`PyInitConfig_GetError` +* :c:func:`PyInitConfig_GetExitCode` * :c:func:`PyInitConfig_HasOption` * :c:func:`PyInitConfig_GetInt` * :c:func:`PyInitConfig_GetStr` @@ -11,7 +13,5 @@ Add functions to configure the Python initialization: * :c:func:`PyInitConfig_SetStr` * :c:func:`PyInitConfig_SetStrList` * :c:func:`Py_InitializeFromInitConfig` -* :c:func:`PyInitConfig_GetError` -* :c:func:`PyInitConfig_GetExitCode` Patch by Victor Stinner. From 43d844b353b52f097c1dc313f5e6a942c9dff0bf Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 00:48:27 +0200 Subject: [PATCH 10/18] revert preconfig.c changes --- Include/internal/pycore_initconfig.h | 4 --- Python/preconfig.c | 52 ++++++++++------------------ 2 files changed, 19 insertions(+), 37 deletions(-) diff --git a/Include/internal/pycore_initconfig.h b/Include/internal/pycore_initconfig.h index 63a38e32a3feaf..25ec8cec3bd249 100644 --- a/Include/internal/pycore_initconfig.h +++ b/Include/internal/pycore_initconfig.h @@ -62,10 +62,6 @@ extern int _PyWideStringList_Copy(PyWideStringList *list, extern PyStatus _PyWideStringList_Extend(PyWideStringList *list, const PyWideStringList *list2); extern PyObject* _PyWideStringList_AsList(const PyWideStringList *list); -extern PyStatus _PyWideStringList_FromBytes( - PyWideStringList *list, - Py_ssize_t length, - char * const *items); /* --- _PyArgv ---------------------------------------------------- */ diff --git a/Python/preconfig.c b/Python/preconfig.c index ebcfa28323f74b..5b26c75de8b3a0 100644 --- a/Python/preconfig.c +++ b/Python/preconfig.c @@ -74,47 +74,33 @@ _Py_COMP_DIAG_POP /* --- _PyArgv ---------------------------------------------------- */ -PyStatus -_PyWideStringList_FromBytes(PyWideStringList *list, - Py_ssize_t length, char * const *items) -{ - PyWideStringList wargv = _PyWideStringList_INIT; - size_t size = sizeof(wchar_t*) * length; - wargv.items = (wchar_t **)PyMem_RawMalloc(size); - if (wargv.items == NULL) { - return _PyStatus_NO_MEMORY(); - } - - for (Py_ssize_t i = 0; i < length; i++) { - size_t len; - wchar_t *arg = Py_DecodeLocale(items[i], &len); - if (arg == NULL) { - _PyWideStringList_Clear(&wargv); - return DECODE_LOCALE_ERR("command line arguments", len); - } - wargv.items[i] = arg; - wargv.length++; - } - - _PyWideStringList_Clear(list); - *list = wargv; - return _PyStatus_OK(); -} - /* Decode bytes_argv using Py_DecodeLocale() */ PyStatus _PyArgv_AsWstrList(const _PyArgv *args, PyWideStringList *list) { + PyWideStringList wargv = _PyWideStringList_INIT; if (args->use_bytes_argv) { - PyStatus status; - status = _PyWideStringList_FromBytes(list, - args->argc, args->bytes_argv); - if (_PyStatus_EXCEPTION(status)) { - return status; + size_t size = sizeof(wchar_t*) * args->argc; + wargv.items = (wchar_t **)PyMem_RawMalloc(size); + if (wargv.items == NULL) { + return _PyStatus_NO_MEMORY(); } + + for (Py_ssize_t i = 0; i < args->argc; i++) { + size_t len; + wchar_t *arg = Py_DecodeLocale(args->bytes_argv[i], &len); + if (arg == NULL) { + _PyWideStringList_Clear(&wargv); + return DECODE_LOCALE_ERR("command line arguments", len); + } + wargv.items[i] = arg; + wargv.length++; + } + + _PyWideStringList_Clear(list); + *list = wargv; } else { - PyWideStringList wargv = _PyWideStringList_INIT; wargv.length = args->argc; wargv.items = (wchar_t **)args->wchar_argv; if (_PyWideStringList_Copy(list, &wargv) < 0) { From 516377489cba8ec489f43d73e93b61cfff12f0d0 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 00:50:46 +0200 Subject: [PATCH 11/18] config_spec_get_member() => config_get_spec_member() --- Python/initconfig.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Python/initconfig.c b/Python/initconfig.c index f16d96937a0a33..9bbd4be141f73a 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -1115,14 +1115,14 @@ PyConfig_SetBytesString(PyConfig *config, wchar_t **config_str, static inline void* -config_spec_get_member(const PyConfigSpec *spec, const PyConfig *config) +config_get_spec_member(const PyConfig *config, const PyConfigSpec *spec) { return (char *)config + spec->offset; } static inline void* -preconfig_spec_get_member(const PyConfigSpec *spec, const PyPreConfig *preconfig) +preconfig_get_spec_member(const PyPreConfig *preconfig, const PyConfigSpec *spec) { return (char *)preconfig + spec->offset; } @@ -1136,8 +1136,8 @@ _PyConfig_Copy(PyConfig *config, const PyConfig *config2) PyStatus status; const PyConfigSpec *spec = PYCONFIG_SPEC; for (; spec->name != NULL; spec++) { - void *member = config_spec_get_member(spec, config); - const void *member2 = config_spec_get_member(spec, (PyConfig*)config2); + void *member = config_get_spec_member(config, spec); + const void *member2 = config_get_spec_member((PyConfig*)config2, spec); switch (spec->type) { case PyConfig_MEMBER_INT: case PyConfig_MEMBER_UINT: @@ -3533,13 +3533,13 @@ initconfig_prepare(PyInitConfig *config, const char *name, void **raw_member) { const PyConfigSpec *spec = initconfig_find_spec(PYCONFIG_SPEC, name); if (spec != NULL) { - *raw_member = config_spec_get_member(spec, &config->config); + *raw_member = config_get_spec_member(&config->config, spec); return spec; } spec = initconfig_find_spec(PYPRECONFIG_SPEC, name); if (spec != NULL) { - *raw_member = preconfig_spec_get_member(spec, &config->preconfig); + *raw_member = preconfig_get_spec_member(&config->preconfig, spec); return spec; } @@ -4016,7 +4016,7 @@ config_get(const PyConfig *config, const PyConfigSpec *spec, } } - void *member = config_spec_get_member(spec, config); + void *member = config_get_spec_member(config, spec); switch (spec->type) { case PyConfig_MEMBER_INT: case PyConfig_MEMBER_UINT: @@ -4223,7 +4223,7 @@ config_set_sys_flag(const PyConfigSpec *spec, int int_value) assert(spec->type == PyConfig_MEMBER_INT || spec->type == PyConfig_MEMBER_UINT || spec->type == PyConfig_MEMBER_BOOL); - int *member = config_spec_get_member(spec, config); + int *member = config_get_spec_member(config, spec); *member = int_value; // Set sys.dont_write_bytecode attribute From 2e37ac4158d470ff7b85b2c6381e75dab7fee92f Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 00:57:08 +0200 Subject: [PATCH 12/18] Check preconfig spec in PyInitConfig_HasOption() --- Programs/_testembed.c | 1 + Python/initconfig.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/Programs/_testembed.c b/Programs/_testembed.c index 8a93633339244a..13f1db1cc708b5 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -1872,6 +1872,7 @@ static int test_initconfig_get_api(void) // test PyInitConfig_HasOption() assert(PyInitConfig_HasOption(config, "verbose") == 1); + assert(PyInitConfig_HasOption(config, "utf8_mode") == 1); assert(PyInitConfig_HasOption(config, "non-existent") == 0); // test PyInitConfig_GetInt() diff --git a/Python/initconfig.c b/Python/initconfig.c index 9bbd4be141f73a..993a5217a04384 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3524,6 +3524,9 @@ int PyInitConfig_HasOption(PyInitConfig *config, const char *name) { const PyConfigSpec *spec = initconfig_find_spec(PYCONFIG_SPEC, name); + if (spec == NULL) { + spec = initconfig_find_spec(PYPRECONFIG_SPEC, name); + } return (spec != NULL); } From 7e3e1aa52b94e804f6b07efb86d4c8328f1ceeca Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 01:05:11 +0200 Subject: [PATCH 13/18] Fix leak in PyInitConfig_GetStrList() + some extra edits --- Doc/c-api/init_config.rst | 3 +++ Python/initconfig.c | 5 +++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 248a7a9e24fd88..ca1e5d6aa9f580 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1682,6 +1682,9 @@ null-terminated UTF-8 encoded string. * Set *\*value*, and return ``0`` on success. * Set an error in *config* and return ``-1`` on error. + *\*value* can be set to ``NULL`` if the option is an optional string and the + option is unset. + On success, the string must be released with ``free(value)``. diff --git a/Python/initconfig.c b/Python/initconfig.c index 993a5217a04384..4b22a715c94ce9 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3631,7 +3631,8 @@ PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) } if (spec->type != PyConfig_MEMBER_WSTR - && spec->type != PyConfig_MEMBER_WSTR_OPT) { + && spec->type != PyConfig_MEMBER_WSTR_OPT) + { initconfig_set_error(config, "config option type is not string"); return -1; } @@ -3676,7 +3677,7 @@ PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, for (Py_ssize_t i=0; i < list->length; i++) { (*items)[i] = wstr_to_utf8(config, list->items[i]); if ((*items)[i] == NULL) { - free(*items); + PyInitConfig_FreeStrList(i, *items); return -1; } } From 1d2725879671a0da2c431863e53329592b4aa607 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 01:07:35 +0200 Subject: [PATCH 14/18] PyInitConfig_SetInt() uses switch/case --- Python/initconfig.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/Python/initconfig.c b/Python/initconfig.c index 4b22a715c94ce9..213c692a07c7aa 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3704,7 +3704,9 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) return -1; } - if (spec->type == PyConfig_MEMBER_INT) { + switch (spec->type) { + case PyConfig_MEMBER_INT: + { if (value < (int64_t)INT_MIN || INT_MAX < (int64_t)value) { initconfig_set_error(config, "config option value is out of int range"); @@ -3714,8 +3716,12 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) int *member = raw_member; *member = int_value; + break; } - else if (spec->type == PyConfig_MEMBER_UINT || spec->type == PyConfig_MEMBER_BOOL) { + + case PyConfig_MEMBER_UINT: + case PyConfig_MEMBER_BOOL: + { if (value < 0 || UINT_MAX < (int64_t)value) { initconfig_set_error(config, "config option value is out of unsigned int range"); @@ -3725,8 +3731,11 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) int *member = raw_member; *member = int_value; + break; } - else if (spec->type == PyConfig_MEMBER_ULONG) { + + case PyConfig_MEMBER_ULONG: + { if (value < 0 || (uint64_t)ULONG_MAX < (uint64_t)value) { initconfig_set_error(config, "config option value is out of unsigned long range"); @@ -3736,8 +3745,10 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) unsigned long *member = raw_member; *member = ulong_value; + break; } - else { + + default: initconfig_set_error(config, "config option type is not int"); return -1; } From e171ca9c9fedff7eec2a8282b63e456ff686e1e4 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 01:14:23 +0200 Subject: [PATCH 15/18] Fix comparisons in PyInitConfig_SetInt() --- Python/initconfig.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Python/initconfig.c b/Python/initconfig.c index 213c692a07c7aa..d2bb46062209da 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -3707,7 +3707,7 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) switch (spec->type) { case PyConfig_MEMBER_INT: { - if (value < (int64_t)INT_MIN || INT_MAX < (int64_t)value) { + if (value < (int64_t)INT_MIN || (int64_t)INT_MAX < value) { initconfig_set_error(config, "config option value is out of int range"); return -1; @@ -3722,7 +3722,7 @@ PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) case PyConfig_MEMBER_UINT: case PyConfig_MEMBER_BOOL: { - if (value < 0 || UINT_MAX < (int64_t)value) { + if (value < 0 || (uint64_t)UINT_MAX < (uint64_t)value) { initconfig_set_error(config, "config option value is out of unsigned int range"); return -1; From d2609c2b7489c4f974a65314554b25054e7b98ec Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 01:21:27 +0200 Subject: [PATCH 16/18] Fix PyInitConfig_GetStr() doc --- Doc/c-api/init_config.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index ca1e5d6aa9f580..f61ca089cd6f67 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1685,7 +1685,8 @@ null-terminated UTF-8 encoded string. *\*value* can be set to ``NULL`` if the option is an optional string and the option is unset. - On success, the string must be released with ``free(value)``. + On success, the string must be released with ``free(value)`` if it's not + ``NULL``. .. c:function:: int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) From 7cecfbdb9c9d565e154f85283e46392918118884 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 01:24:17 +0200 Subject: [PATCH 17/18] Reformat header file --- Include/cpython/initconfig.h | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/Include/cpython/initconfig.h b/Include/cpython/initconfig.h index 053b533cee74ee..328828a9152916 100644 --- a/Include/cpython/initconfig.h +++ b/Include/cpython/initconfig.h @@ -283,25 +283,32 @@ typedef struct PyInitConfig PyInitConfig; PyAPI_FUNC(PyInitConfig*) PyInitConfig_Create(void); PyAPI_FUNC(void) PyInitConfig_Free(PyInitConfig *config); -PyAPI_FUNC(int) PyInitConfig_GetError(PyInitConfig* config, const char **err_msg); -PyAPI_FUNC(int) PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode); - -PyAPI_FUNC(int) PyInitConfig_HasOption(PyInitConfig *config, const char *name); -PyAPI_FUNC(int) PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value); -PyAPI_FUNC(int) PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value); -PyAPI_FUNC(int) PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items); +PyAPI_FUNC(int) PyInitConfig_GetError(PyInitConfig* config, + const char **err_msg); +PyAPI_FUNC(int) PyInitConfig_GetExitCode(PyInitConfig* config, + int *exitcode); + +PyAPI_FUNC(int) PyInitConfig_HasOption(PyInitConfig *config, + const char *name); +PyAPI_FUNC(int) PyInitConfig_GetInt(PyInitConfig *config, + const char *name, + int64_t *value); +PyAPI_FUNC(int) PyInitConfig_GetStr(PyInitConfig *config, + const char *name, + char **value); +PyAPI_FUNC(int) PyInitConfig_GetStrList(PyInitConfig *config, + const char *name, + size_t *length, + char ***items); PyAPI_FUNC(void) PyInitConfig_FreeStrList(size_t length, char **items); -PyAPI_FUNC(int) PyInitConfig_SetInt( - PyInitConfig *config, +PyAPI_FUNC(int) PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value); -PyAPI_FUNC(int) PyInitConfig_SetStr( - PyInitConfig *config, +PyAPI_FUNC(int) PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char *value); -PyAPI_FUNC(int) PyInitConfig_SetStrList( - PyInitConfig *config, +PyAPI_FUNC(int) PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items); From 5ae1b6976332d737d7f9e7881a511cdf92fe1b91 Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Tue, 3 Sep 2024 14:04:57 +0200 Subject: [PATCH 18/18] Update doc --- Doc/c-api/init_config.rst | 2 +- Doc/whatsnew/3.14.rst | 5 +++-- .../C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index f61ca089cd6f67..b59be98608a158 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1642,7 +1642,7 @@ Error Handling Get the *config* exit code. - * Set *\*exitcode* and return ``1`` if Python wants to exit. + * Set *\*exitcode* and return ``1`` if *config* has an exit code set. * Return ``0`` if *config* has no exit code set. Only the ``Py_InitializeFromInitConfig()`` function can set an exit diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index f2a73ff61728f1..997d51ce798aa0 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -492,7 +492,8 @@ New Features * Add :c:func:`Py_HashBuffer` to compute and return the hash value of a buffer. (Contributed by Antoine Pitrou and Victor Stinner in :gh:`122854`.) -* Add functions to get and set the current runtime Python configuration: +* Add functions to get and set the current runtime Python configuration + (:pep:`741`): * :c:func:`PyConfig_Get` * :c:func:`PyConfig_GetInt` @@ -501,7 +502,7 @@ New Features (Contributed by Victor Stinner in :gh:`107954`.) -* Add functions to configure the Python initialization: +* Add functions to configure the Python initialization (:pep:`741`): * :c:func:`PyInitConfig_Create` * :c:func:`PyInitConfig_Free` diff --git a/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst b/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst index 8496b91f183d25..370e2690c458ba 100644 --- a/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst +++ b/Misc/NEWS.d/next/C_API/2024-08-30-14-02-17.gh-issue-107954.TPvj4u.rst @@ -1,4 +1,4 @@ -Add functions to configure the Python initialization: +Add functions to configure the Python initialization (:pep:`741`): * :c:func:`PyInitConfig_Create` * :c:func:`PyInitConfig_Free`