PEP 739 / `build-details.json`: `c_api.headers` does not include the `pythonX.Y` directory

[mgorny]

Bug report

Bug description:

Sorry I didn't notice this while fixing #134455 but c_api.headers key contains the path to top-level system include directory, i.e.:

{
  "c_api": {
    "headers": "/usr/local/include",
    "pkgconfig_path": "/usr/local/lib/pkgconfig"
  }
}

while according to the examples in PEP 739, it should contain the path to the pythonX.Y directory:

Examples

/usr/include/python3.14
include/python3.14
etc.

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-134632: Fix build-details.json to use INCLUDEPY path #134633
• [3.14] gh-134632: Fix build-details.json to use INCLUDEPY path (GH-134633) #135605
• gh-134632: Add iOS/Android test skip for C API check for headers. #135656
• [3.14] gh-134632: Add iOS/Android test skip for C API check for headers. (GH-135656) #135657

[encukou]

This broke Android & iOS, see e.g. here:

======================================================================
FAIL: test_c_api (test.test_build_details.CPythonBuildDetailsTests.test_c_api)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/0FEBFE27-CF48-433B-AF78-485F97B08C80/data/Containers/Bundle/Application/BC1C3834-4B74-4BFC-8F77-9D59306D4B53/iOSTestbed.app/python/lib/python3.15/test/test_build_details.py", line 129, in test_c_api
    self.assertTrue(os.path.exists(os.path.join(value['headers'], 'Python.h')))
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: False is not true

[mgorny]

FWIU this means that build-details.json was broken on these platforms in the first place. Could you give me more details, i.e. what build-details.json contains, where Python.h is installed, and how to get that path?

[encukou]

@freakboy3742, do you have that info?

[encukou]

I doubt the headers get installed after cross-compilation, but, let's wait for the experts :)

[freakboy3742]

@encukou The headers aren't installed into a running app on iOS or Android, so I think this test may need to be a skip (at least in the short term).

The headers will exist for a development environment though. However, as I flagged during the recent Core Team summit, the location isn't fixed - it essentially can't be "baked in" at time of compilation, because the install location could be wherever the user has downloaded/installed the distribution.

So - the short term fix will be a test skip; longer term, we need to determine what build-details.json should contain in a relocatable environment. I guess what we need here is the equivalent of $RPATH handling, but for the location of a relocatable Python install.

[itamaro]

we need to determine what build-details.json should contain in a relocatable environment

iiuc, PEP-739 already allows relative paths to base_prefix, and base_prefix itself can be relative to the location of the file, so supposedly this should already be relocatable-friendly, no?