Promote `PyUnstable_Object_GC_NewWithExtraData()` to the stable API

[wjakob]

Feature or enhancement

Proposal:

Allocating variable sized Python objects via the C API is a genuinely useful operation. It is e.g. needed to

• implement custom data structures (tuples, lists)
• implement binding libraries (like nanobind) that co-locate binding data structures with Python objects

So far, the standard way to create them was via PyVarObject, and this was supported even in stable ABI builds. Unfortunately, this data structure becomes inaccessible with the upcoming free-threaded flavor of the stable ABI.

I would like to propose that the function PyUnstable_Object_GC_NewWithExtraData() be promoted to the stable API to provide an escape hatch for extensions that require variable sized objects in abi3t. This will enable extensions to implement variable sized objects without depending on the specifics of PyVarObject.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-148498: Promote PyUnstable_Object_GC_NewWithExtraData() to the stable API #148499

[zooba]

The fact that we made it an unstable API in the first place speaks very strongly against making it a stable API. I think we should not do this, and instead look at the underlying need and design a proper API that allows us to adapt the underlying implementation without users being affected.