gh-111506: Implement Py_SET_REFCNT() as opaque function in limited C API

[vstinner]

In the limited C API version 3.13, Py_SET_REFCNT() function is now implemented as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.

• Issue: --disable-gil builds are not compatible with the limited API #111506

[encukou]

Why is this particular function necessary?
I'd rather not commit to supporting setting the reference count to an arbitrary value in the long term.

abi3 makes it possible -- it exposes ob_refcnt directly. But exposing the fields directly was perhaps the biggest flaw of abi3. We now have a chance to revisit it, and tighten the set of supported operations.

[vstinner]

Why is this particular function necessary?

It's needed to be able to remove non-portable assembly code from Include/*.h. I would prefer to keep these header files as simple C code, no inline assembly.

The code was just change for free threading. This change follows Py_INCREF/DECREF which already use an opaque function call in limited C API version 3.12.

[vstinner]

I'd rather not commit to supporting setting the reference count to an arbitrary value in the long term.

I also have concerns about that. But this change is a practical change to make the situation less bad.

I plan to work on a whole PEP to disallow accessing directly PyObject.ob_refcnt and convert more functions to opaque function calls in the limited C API, as Sam describes in the issue.

[vstinner]

@colesbury @serhiy-storchaka @erlend-aasland: Would you mind to review this change?

[erlend-aasland]

I'll leave this to @colesbury.

[colesbury]

LGTM

I think this only improves our compatibility story going forward. We have already committed to this functionality as part of the API and ABI. Using an opaque function instead of an inline function does not change this.

[encukou]

+1 API-wise -- keeping it abi_only is the way to go.

[vstinner]

Merged, thanks for reviews.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora 3.x has failed when building commit 20cfab9.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/223/builds/4734) and take a look at the build logs.
4. Check if the failure is related to this commit (20cfab9) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/223/builds/4734

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 17, done.        
remote: Counting objects:   7% (1/13)        
remote: Counting objects:  15% (2/13)        
remote: Counting objects:  23% (3/13)        
remote: Counting objects:  30% (4/13)        
remote: Counting objects:  38% (5/13)        
remote: Counting objects:  46% (6/13)        
remote: Counting objects:  53% (7/13)        
remote: Counting objects:  61% (8/13)        
remote: Counting objects:  69% (9/13)        
remote: Counting objects:  76% (10/13)        
remote: Counting objects:  84% (11/13)        
remote: Counting objects:  92% (12/13)        
remote: Counting objects: 100% (13/13)        
remote: Counting objects: 100% (13/13), done.        
remote: Compressing objects:  10% (1/10)        
remote: Compressing objects:  20% (2/10)        
remote: Compressing objects:  30% (3/10)        
remote: Compressing objects:  40% (4/10)        
remote: Compressing objects:  50% (5/10)        
remote: Compressing objects:  60% (6/10)        
remote: Compressing objects:  70% (7/10)        
remote: Compressing objects:  80% (8/10)        
remote: Compressing objects:  90% (9/10)        
remote: Compressing objects: 100% (10/10)        
remote: Compressing objects: 100% (10/10), done.        
remote: Total 17 (delta 3), reused 3 (delta 3), pack-reused 4        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '20cfab903db70cf952128bc6b606e3ec4a216498'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 20cfab903d gh-111506: Implement Py_SET_REFCNT() as opaque function in limited C API (#111508)
Switched to and reset branch 'main'

make: *** [Makefile:2065: buildbottest] Error 3