Skip to content

Pixi rebuilds all 36 Cython extensions on every pixi run (editable install temp dirs) #2138

@rparolin

Description

@rparolin

Symptom

Every invocation of pixi run -e cu13 python examples/<foo>.py rebuilds all 36 Cython extensions in cuda_bindings (~27s of C++ compilation work per run), even when nothing has changed.

Build output shows a fresh /tmp/tmpXXXXXXXX.build-temp/ and .build-lib/ directory each run.

Root cause (3 layers)

  1. pixi-build: sees cuda_bindings as a path dep and unconditionally runs uv pip install --editable ../cuda_bindings on every pixi run, even when nothing changed.

  2. setuptools: editable_wheel._create_wheel_file (editable_wheel.py:342-344) hardcodes:

    build_lib = TemporaryDirectory(suffix=".build-lib")
    build_tmp = TemporaryDirectory(suffix=".build-temp")

    These are brand-new empty dirs every time.

  3. build_ext: walks the 36 Cython extensions and asks "is the .so in build_lib newer than the .cpp source?" Since build_lib is fresh and empty, the answer is always "no" → recompile everything. 36 × ~750ms ≈ 27s per run.

Why the fix is non-trivial

Simply pinning the build dirs to a fixed location would change behavior for CI configurations running in parallel. A reasonable fix likely involves a build-system toggle (env var or pixi option) so local dev machines can opt into a stable build dir while CI keeps the current isolated behavior.

Possible workaround to investigate

Setting no-build-isolation = true under [pypi-options] in the pixi manifest may help. See: https://pixi.prefix.dev/latest/reference/pixi_manifest/#no-build-isolation

Metadata

Metadata

Assignees

Labels

triageNeeds the team's attention

Type

No fields configured for Bug.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions