Skip to content

gh-117953: Track All Extension Modules #118156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 44 commits into
base: main
Choose a base branch
from
Draft

gh-117953: Track All Extension Modules #118156

wants to merge 44 commits into from
+931 −319

Conversation

ericsnowcurrently
Copy link
Member

@ericsnowcurrently ericsnowcurrently commented Apr 22, 2024 •
edited by bedevere-app bot
Loading

When an extension module is loaded, the init function is looked up and run. If that doesn't fail then the module is "fixed up", which includes adding its PyModuleDef to a global mapping that is used if the module is re-loaded. However, that has not applied to multi-phase init modules. Adding those modules to the global mapping would identify that we already loaded the module (or at least tried), as well as the module's kind. That's especially useful for a follow up PR I have planned.

(This is based on gh-118116. For the diff, see my track-all-ext-modules branch.)

@ericsnowcurrently
Factor out _PyImport_RunDynamicModule().
cc67849
@ericsnowcurrently
Drop _PyImport_LoadDynamicModuleWithSpec().
2e5ecde
@ericsnowcurrently
Add an assert.
b810cd8
@ericsnowcurrently
Factor out fix_up_extension_for_interpreter().
8d0ef38
@ericsnowcurrently
Do _extensions_cache_set() first.
7f930cc
@ericsnowcurrently
Calculate path_bytes ahead of time.
1ffeaef
@ericsnowcurrently
Move the audit entry to _imp_create_dynamic_impl().
7500285
@ericsnowcurrently
_Py_ext_module_loader_info_from_spec() -> _Py_ext_module_loader_info_…
3c0b170 
…init_from_spec()
@ericsnowcurrently
Add info.newcontext.
02e24cc
@ericsnowcurrently
Split up _PyImport_RunDynamicModule() along cross-interpreter-safe bo…
ba5ffa7 
…undaries.
@ericsnowcurrently
Drop _PyImport_RunDynamicModule().
b895966
@ericsnowcurrently
Always close the file handle.
76fa273
@ericsnowcurrently
Revert the fclose() change.
1e936c5
@ericsnowcurrently
Factor out create_dynamic().
6bb07b9
@ericsnowcurrently
Drop _PyImport_FixupExtensionObject().
2705e65
@ericsnowcurrently
Move setting __file__ to fix_up_extension().
98d8b4b
@ericsnowcurrently
Pass tstate through to fix_up_extension().
ddf1135
@ericsnowcurrently
Add a note about _PyImport_FixupBuiltin().
694a763
@ericsnowcurrently
Always expect a valid modules dict passed to fix_up_extension().
72e0d73
@ericsnowcurrently
Factor out reload_singlephase_extension().
3f14a67
@ericsnowcurrently
res.singlephase -> res.kind
b5f5868
@ericsnowcurrently
Factor out _extensions_cache_init().
a5f7535
@ericsnowcurrently
Factor out _extensions_cache_find_unlocked().
e8d674a
@ericsnowcurrently
Use _PyImport_RunModInitFunc() in create_builtin().
27b365d
@ericsnowcurrently
Clean up _PyImport_FixupBuiltin().
5128702
@ericsnowcurrently
Add some asserts to _PyImport_RunModInitFunc().
aaec360
@ericsnowcurrently
Drop the replace arg to _extensions_cache_set().
f97671e
@ericsnowcurrently
Check the module returned by import_find_extension() to ensure single…
2366983 
…-phase init.
@ericsnowcurrently
Allow mod->md_def to be NULL.
d5039ea
@ericsnowcurrently
Call _PyImport_RunModInitFunc() in reload_singlephase_extension().
67d217b
@ericsnowcurrently
res->err is an array, not just a pointer.
9e40287
@ericsnowcurrently
Factor out import_run_extension() (and drop create_dynamic()).
b8c2a1d
@ericsnowcurrently
Factor out update_extension_cache() and drop fix_up_extension_for_int…
967c649 
…erpreter().
@ericsnowcurrently
We already know the def is okay.
aecf56c
@ericsnowcurrently
Revive fix_up_extension_for_interpreter().
0d03d23
@ericsnowcurrently
Add a distinct _Py_ext_module_loader_info.path (and old path -> filen…
cace102 
…ame).
@ericsnowcurrently
Do not call update_extensions_cache() in fix_up_extension().
3cf0db0
@ericsnowcurrently
Let the modules arg to fix_up_extension_for_interpreter() be NULL.
9b3088a
@ericsnowcurrently
Let the caller of update_extensions_cache() decide if m_copy should b…
4ac1644 
…e populated.
@ericsnowcurrently
Fix the update_extensions_cache() signature.
393a177
@ericsnowcurrently
Imply singlephase if the init func fails.
2969e4c
@ericsnowcurrently
Keep track of all module defs.
93c7f79
@ericsnowcurrently
Fix the implementation.
cb2137e
@ericsnowcurrently
wip
895ce51
@bedevere-app bedevere-app bot mentioned this pull request Apr 22, 2024
Open
@ericsnowcurrently ericsnowcurrently mentioned this pull request Apr 22, 2024
Merged
@ericsnowcurrently ericsnowcurrently marked this pull request as draft May 7, 2024 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 will be requested when the pull request is marked ready for review kumaraditya303 is a code owner

@brettcannon brettcannon Awaiting requested review from brettcannon brettcannon will be requested when the pull request is marked ready for review brettcannon is a code owner

@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan will be requested when the pull request is marked ready for review ncoghlan is a code owner

@warsaw warsaw Awaiting requested review from warsaw warsaw will be requested when the pull request is marked ready for review warsaw is a code owner

Assignees
No one assigned
Labels
skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@ericsnowcurrently