Support modification of builtins module.

[mrolle45]

Feature

Support modification of builtins module. That is, the one called <module 'builtins' (built-in)>.
I don't propose use of the name __builtins__ because that is implementation dependent.

Pitch

Why would anyone want to alter builtins?

One use case would be a program involving several modules, and you want to make a name, like foo visible in all of them without having to explicitly import the name from some common module (or explicitly set it globally) in each of those modules.

In one of the modules, perhaps main, you would have

import builtins
builtins.foo = 42

Alternatively, you could have:
sys.modules['builtins].foo = 42

Then in every module, the name foo would have the value 42, just like int has the value <class int>.

Or would it always??

Any time the interpreter looks up the name foo as a global name, and it is not in the module's globals, it will look in the builtins module as it exists at that time. After the module has been imported, then certainly the builtins module is now in its modified state. But during the import of a module, builtins may or may not have been modified yet.

So what's mypy to do?

I think a reasonable approach would be to treat the modification as being visible everywhere, just like any other object which is visible as the same object in several modules' global namespaces through imports. It could be considered as a nameless member of the module's global namespace. That is, multiple assignments to the attribute foo could be checked for compatible types, and the first one that mypy encounters can be used to infer the type.

If you want to go farther into analyzing where in the import of a module something happens, it's the same deal as any other object that might be imported from somewhere else. This is easy enough to resolve, unless there are import cycles involved. I've discussed problems with import cycles in some other issues I have posted, but the gist of it is that mypy would need to see if the identity of a name would vary depending on the order of actual imports at runtime, and any ambiguity ought to be considered an error.

[mrolle45]

I'm working on some code that will analyze all names appearing in the collection of modules, prior to semantic or type analysis of any of those modules. It will identify the object being referred to by every occurrence of a name, taking into account imports (including cyclic imports) and name resolution rules. I will leave a hook for the builtins module, populating it from typeshed. This could be used by mypy to record changes made to builtins.

[erictraut]

Dynamically modifying the namespace of other modules goes against the whole notion of type checking and type safety. This is especially true for builtins. If you care about code maintainability, robustness, and module composability, it is something I would highly discourage.

That said, there are legitimate uses for declaring new (non-standard) symbols in the builtins namespace. In particular, there are alternate Python distros for specialized use cases and environments that include additional builtins symbols.

We decided to support this in pyright through the use of a specially-named __builtins__.pyi stub file. If a stub with this name is found within the root directory of a project, pyright treats any symbols declared within that stub as though they are appended to the bottom of builtins.pyi.

If that sounds like a good solution to the problem you've posed, perhaps mypy could adopt this same approach.

[mrolle45]

I started a discussion on Python Ideas.
This would be a straightforward way of adding names to builtins in standard Python, and would be applied to specific places in the package/module hierarchy. It's similar to your scheme with pyright, and I gave you credit for the idea. It uses a .py module rather than a .pyi stub, and can be applied any number of places, including nested, in the hierarchy.
I hope we'll see this in Python 3.17 or whatever.

[mrolle45]

If that sounds like a good solution to the problem you've posed, perhaps mypy could adopt this same approach.

Yes and no.
Yes, that would be suitable for alternate Python distros, where the additional symbols are statically part of the environment. For these distros, are the added symbols hard baked in, or can the user choose added symbols on a per project basis (with some mechanism in the distro to generate the needed stub file).
No, I posed a different situation where I want to run my program with standard Python, and dynamically add the new symbols. I have to do two things.

1. Get the new symbols into builtins, with Python code running in my project. Ideally, this happens before anything else in the project gets loaded.
2. Telling mypy about the new symbols.

This could be deplored as untypesafe behavior, unless there were a way, with standard Python, to get these extra names added to builtins at the time that builtins is first built in the interpreter. Perhaps a special file name like builtins.py in the project root, and a way of rigging the import system so that importing 'builtins' will invoke some special loader (perhaps).
Maybe the best way to provide this capability is to build a mechanism into the standard python release that allows for customizing the builtins module. Then mypy could see if that occurs in a particular project and update its view of builtins accoringly.

[mrolle45]

@erictraut Help! I made an edit to the aforementioned discussion page, and its spam filter decided to hide it. Can you do anything to unfreeze it or tell me how I can do it for myself?

[mrolle45]

Actually, I just got email saying that my post is being reviewed and should reappear soon. So never mind the above.
Later, I got another email and the discussion is back.

[erictraut]

Dynamically injecting new symbols into another module's namespace — especially the builtins module — is highly problematic. The same can be said of dynamically injecting new attributes into a class or object. It goes against core principles of software engineering: modularity, composability, separation of concerns, etc. This is an anti-pattern, not something that should be standardized and encouraged.

What is the benefit of doing this? To save an import statement?

[AlexWaygood]

Maybe the best way to provide this capability is to build a mechanism into the standard python release that allows for customizing the builtins module.

Maybe — but that's well outside the purview of mypy, which is a third-party type checker independent of CPython. Until such a time as CPython makes these changes (I think it's unlikely to ever make these changes), I don't think it's reasonable to expect mypy to support the kind of thing you're looking for here.

I'm closing this issue — if anybody wants to request a mypy equivalent of the more limited feature Eric's introduced over at pyright, they can open another issue to do so. For the feature request you're making, @mrolle45, I don't think there's any realistic chance of mypy ever supporting dynamic modifications of builtins, I'm afraid.

[emmatyping]

Another possible work-around would be to write a plugin for mypy, but I agree we are very unlikely to support this use case