From 0993d40d59df5117d4476284a8163d8d4ae72e87 Mon Sep 17 00:00:00 2001 From: Enrique Saurez Date: Fri, 29 May 2026 17:49:57 -0700 Subject: [PATCH] [nanvix] E: Bake ninja and Cython into toolchain image MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds `ninja-build`, `python3-pip`, and `Cython<3` to the toolchain-python docker image so that meson- and Cython-based Python extension cross-builds (numpy, scipy, ...) work out-of-the-box, without an apt/pip preamble on every `docker run` invocation. What changed in `.nanvix/docker/Dockerfile`: - Added `python3-pip` and `ninja-build` to the apt install list. - Added `pip3 install --break-system-packages 'Cython<3'` (pinned for numpy 1.26.x compatibility; lift the pin when bumping numpy). - Added `rm -rf /usr/include/python3.12` after the install. The `python3-pip`/`ninja-build` apt packages transitively pull in `libpython3.12-dev`, whose headers under `/usr/include/python3.12` would otherwise be picked up by meson's regen step ahead of the Nanvix cross sysroot headers and silently corrupt the cross-build. - Comment block explaining the rationale for each addition and the `/usr/include/python3.12` purge. Why this matters: The numpy `.so` cross-build (validated end-to-end on 2026-05-27 with the STB_WEAK loader fix landed) requires two tools that were not present in the image as shipped: - `ninja` — meson's default backend; missing it makes every meson-based Python extension build fail immediately. - `Cython` — used by `numpy/_build_utils/tempita.py` to template `.pyx.in` files; without it the `numpy.random` codegen step fails. Before this change, the workaround was to inject: ```bash apt-get update -qq apt-get install -qq -y --no-install-recommends ninja-build python3-pip pip3 install --quiet --break-system-packages 'Cython<3' rm -rf /usr/include/python3.12 ``` into every numpy build invocation, which (a) was fragile, (b) required the docker container to have outbound network access on every build (non-hermetic), and (c) re-paid the apt install cost in CI every run. Validated locally: - `docker build -f .nanvix/docker/Dockerfile -t toolchain-python:pr13 .nanvix/docker/` succeeds. - `docker run --rm bash -c 'ninja --version'` → `1.11.1`. - `docker run --rm bash -c 'python3 -c "import Cython; print(Cython.__version__)"'` → `0.29.37`. - `docker run --rm bash -c 'ls /usr/include/python3.12'` → exits non-zero / "No such file or directory". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .nanvix/docker/Dockerfile | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/.nanvix/docker/Dockerfile b/.nanvix/docker/Dockerfile index 8edab3fd54399fa..5d3dab2a4dd565b 100644 --- a/.nanvix/docker/Dockerfile +++ b/.nanvix/docker/Dockerfile @@ -6,9 +6,28 @@ FROM ghcr.io/nanvix/toolchain-gcc:sha-34a3641 +# Install the host Python plus the build helpers required by extension +# modules that ship a meson or Cython build step (numpy, scipy, pandas, +# ...): +# +# - ninja — meson's default backend; missing it makes every meson-based +# extension build fail immediately. +# - Cython — required by numpy 1.26.4's `numpy/_build_utils/tempita.py` +# .pyx.in code generation. Pinned `<3` for numpy 1.26.x +# compatibility; lift the pin when bumping numpy. +# +# We deliberately purge `/usr/include/python3.12` after the install. The +# `python3-pip` / `ninja-build` apt packages transitively pull in +# `libpython3.12-dev`, whose headers under `/usr/include/python3.12` would +# otherwise be picked up by meson's regen step ahead of the Nanvix cross +# sysroot headers and silently corrupt the cross-build. RUN apt-get update \ && apt-get install -y --no-install-recommends \ python3 \ python3-dev \ + python3-pip \ + ninja-build \ + && pip3 install --break-system-packages --no-cache-dir 'Cython<3' \ + && rm -rf /usr/include/python3.12 \ && rm -rf /var/lib/apt/lists/* \ && ln -sf /usr/bin/python3 /opt/nanvix/bin/python3