NotImplementedError raised by annotationlib (?) when importing classes derived from NamedTuple

[befeleme]

Bug report

Bug description:

With 3.14.0a3 there is an issue which affects all of the building block packages: setuptools, pip, wheel, causing failure to build of over 2800 packages in Fedora Linux total.

In cases I explored, the issue appears when importing classes derived from NamedTuple.
To reproduce in the Python 3.14.0a3 interpreter:

>>> import setuptools.package_index
Traceback (most recent call last):
  File "<python-input-18>", line 1, in <module>
    import setuptools.package_index
  File "/usr/lib/python3.14/site-packages/setuptools/package_index.py", line 1005, in <module>
    class Credential(NamedTuple):
    ...<12 lines>...
            return f'{self.username}:{self.password}'
  File "/usr/lib64/python3.14/typing.py", line 2971, in __new__
    types = annotationlib.call_annotate_function(original_annotate, annotationlib.Format.FORWARDREF)
  File "/usr/lib64/python3.14/annotationlib.py", line 613, in call_annotate_function
    result = func(Format.VALUE_WITH_FAKE_GLOBALS)
  File "/usr/lib/python3.14/site-packages/setuptools/package_index.py", line 1005, in __annotate__
    class Credential(NamedTuple):
    
NotImplementedError

The class in question looks like this: https://github.com/pypa/setuptools/blob/main/setuptools/package_index.py#L1003

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-128089: Bump the PYC magic number #128097

[tomasr8]

cc @JelleZijlstra

[vstinner]

What is your setuptools version? I cannot reproduce the issue on the main branch with setuptools 75.6.0.

[vstinner]

I failed to reproduce the issue on Python 3.14 alpha4 with setuptools 75.6.0:

wget https://www.python.org/ftp/python/3.14.0/Python-3.14.0a3.tar.xz
tar -xf Python-3.14.0a3.tar.xz 
cd Python-3.14.0a3/
./configure --prefix /opt/py314 && make && make install
/opt/py314/bin/python3 -m pip install setuptools
/opt/py314/bin/python3  -c 'import setuptools.package_index'

[DavidCEllis]

I can't reproduce this either on 3.14.0a3. It looks like something is out of sync or hasn't rebuilt?

The values of the Format Enum changed from 3.14.0a2 to 3.14.0a3. In a2, the value '2' meant Format.FORWARDREF, in a3 this is now Format.VALUE_WITH_FAKE_GLOBALS. (FORWARDREF is now 3).

Calling 'func' with FORWARDREF would give this NotImplementedError, and so calling it with Format.VALUE_WITH_FAKE_GLOBALS would also do so if something hasn't updated (for example you would see this if you used annotationlib from 3.14.0a3 on a build of 3.14.0a2).

[frenzymadness]

Is it possible that when the module is cached into pyc file (during our RPM builds) the numeric value under annotationlib.Format.* is stored as the respective integer somewhere in the code object of __annotate__ function? And then, when we run the code from the pyc file with newer Python, it expects the new value there but the old one is stored from the build time, right?

I've tried to find the value stored somewhere but did not succeed so I'm asking whether the approach was correct or the problem is completely different.

[DavidCEllis]

I think it's the other way around. Your annotationlib is up to date, but something about the runtime is outdated?

Can you try creating a class and calling its __annotate__ function with the value 2 and seeing what the output is?

EG:

class X:
    a: int

X.__annotate__(2)

See if you get the same NotImplementedError. In 3.14.0a3 this should return the annotation dict, in 3.14.0a2 you'll get the error.

---

3.14.0a2

>>> class X:
...     a: int
...
>>> X.__annotate__(2)
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    X.__annotate__(2)
    ~~~~~~~~~~~~~~^^^
  File "<python-input-0>", line 1, in __annotate__
    class X:

NotImplementedError

3.14.0a3

>>> class X:
...     a: int
...
>>> X.__annotate__(2)
{'a': <class 'int'>}

This is due to the enum change:

The enum values:

3.14.0a2

>>> import annotationlib
>>> list(annotationlib.Format)
[<Format.VALUE: 1>, <Format.FORWARDREF: 2>, <Format.STRING: 3>]

3.14.0a3

>>> import annotationlib
>>> list(annotationlib.Format)
[<Format.VALUE: 1>, <Format.VALUE_WITH_FAKE_GLOBALS: 2>, <Format.FORWARDREF: 3>, <Format.STRING: 4>]

[befeleme]

It looks there was something out of sync: I rebootstrapped the whole environment and don't see the issue in the newly built packages. It only appears if anything in the test environment runs the NamedTuple code path in packages that were built with alpha2. Our Python behaves correctly as alpha3 according to the provided code snippets.

[vstinner]

Ok, I reproduced the issue. IMO we should bump PYC serial number

Code bug.py:

from typing import NamedTuple

class Credential(NamedTuple):
    username: str
    password: str
    def __str__(self) -> str:
        return f'{self.username}:{self.password}'

Steps to reproduce:

• Install Python 3.14 alpha2 (ex: install in /opt/py314a2)
• Install Python 3.14 alpha3 (ex: install in /opt/py314)
• Create the PYC file using alpha2: /opt/py314a2/bin/python3 -c 'import bug'
• Run the script using alpha3, using alpha2 PYC: /opt/py314/bin/python3 -c 'import bug'

Output:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import bug
  File "/home/vstinner/bug/bug.py", line 3, in <module>
    class Credential(NamedTuple):
    ...<3 lines>...
            return f'{self.username}:{self.password}'
  File "/opt/py314/lib/python3.14/typing.py", line 2971, in __new__
    types = annotationlib.call_annotate_function(original_annotate, annotationlib.Format.FORWARDREF)
  File "/opt/py314/lib/python3.14/annotationlib.py", line 613, in call_annotate_function
    result = func(Format.VALUE_WITH_FAKE_GLOBALS)
  File "/home/vstinner/bug/bug.py", line 3, in __annotate__
    class Credential(NamedTuple):
    
NotImplementedError

[vstinner]

IMO we should bump PYC serial number

The problem is that Python 3.14 alpha3 is already released. I'm not sure if it's still relevant to bump the serial number afterwards?

[JelleZijlstra]

Sorry for this, I should indeed have bumped the magic number. I sort of assumed it was harmless to not bump the magic number for every individual change in the alpha phase. Maybe it would be easier to just bump the number at every alpha release and not worry about it for individual changes.

[DavidCEllis]

Might be worth bumping it before 3.14a4 - to avoid anyone running into this if they've skipped a3?

I didn't realise the generated __annotate__ function was stored in the .pyc but this makes sense now.