Skip to content

[nanvix] E: Build bz2/lzma/zlib/sqlite3 as DT_NEEDED .so chains#739

Open
esaurez wants to merge 1 commit into
feat/wave5-pr-b-unbundle-externalsfrom
feat/wave5-pr-c-unbundle-group-a
Open

[nanvix] E: Build bz2/lzma/zlib/sqlite3 as DT_NEEDED .so chains#739
esaurez wants to merge 1 commit into
feat/wave5-pr-b-unbundle-externalsfrom
feat/wave5-pr-c-unbundle-group-a

Conversation

@esaurez

@esaurez esaurez commented Jun 17, 2026

Copy link
Copy Markdown

Summary

Completes the external-library unbundling started in #738 by moving the remaining four stdlib extensions (_bz2, _lzma, zlib, _sqlite3) from being statically linked into python.elf to runtime-loaded .so files emitting DT_NEEDED for libbz2.so / liblzma.so / libz.so / libsqlite3.so.

After this PR, python.elf bundles no external Nanvix-ported libraries at all — every .so wrapper (_ssl, _hashlib, _ctypes, _sqlite3, zlib, _bz2, _lzma) consumes its underlying library through DT_NEEDED. This matches upstream cpython's default behavior on every Linux distro (system-library detection enabled).

Why

The previous PR (#738) took the libffi / openssl / lxml consumers to DT_NEEDED .so chains but left these four extensions statically linked into python.elf because the Nanvix port repos for bzip2 / zlib / xz / sqlite did not yet ship .so builds. The four port-repo PRs listed under Dependencies below add those .so builds, which finally lets this PR finish the migration.

What changed

4 stdlib modules migrated from *static* (linked into python.elf) to *shared* (loaded via dlopen). The corresponding LIBSQLITE3_LIBS / ZLIB_LIBS / BZIP2_LIBS / LIBLZMA_LIBS env vars already use -L/-l form in #738 (no change needed there); cpython's normal MODULE_*_LDFLAGS machinery wires them into each .so's link line as DT_NEEDED.

DT_NEEDED .so chains added by this PR:
  _bz2.cpython-312.so       -> libbz2.so
  _lzma.cpython-312.so      -> liblzma.so
  zlib.cpython-312.so       -> libz.so
  _sqlite3.cpython-312.so   -> libsqlite3.so

The Nanvix dynamic loader walks each DT_NEEDED chain at dlopen() time, and UND symbols in any of those .so files bind to python.elf .dynsym.

Mechanics

File Change
.nanvix/setup_local.py Adds 4 *shared* entries (_bz2, _lzma, zlib, _sqlite3).
Makefile.nanvix Drops -lsqlite3 -lz -lbz2 -llzma from python.elf LIBS; the -Wl,--start-group ... -Wl,--end-group block is gone. Per-module *_LIBS env vars unchanged (already in -L/-l form from #738).
.nanvix/runtime_sos.py REQUIRED_RUNTIME_SOS gains libbz2.so / liblzma.so / libz.so / libsqlite3.so so .nanvix/test.py and .nanvix/package.py stage them into sysroot/lib/ at run time.
.nanvix/z.py _DEP_EXPECTED_LIBS for bzip2, zlib, sqlite, xz extended with the .so siblings so ./z setup validates that the new shared-library port builds installed correctly into buildroot/lib.
.nanvix/test.py Extends CPYTHON_TEST_EXTERNAL_DEPS smoke test with the 4 new modules.

Dependencies

Base branch: #738.

Runtime dependencies — the four port-repo PRs that produce the .so files this PR consumes:

These four PRs must land and their port-lib releases must be pinned in cpython's nanvix.toml before this PR is mergeable upstream.

Also inherits the runtime deps already required by #738:

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Completes the external-library unbundling started in the parent PR by
moving the remaining four stdlib extensions (_bz2, _lzma, zlib,
_sqlite3) from being statically linked into python.elf to runtime-
loaded .so files that emit DT_NEEDED for libbz2.so / liblzma.so /
libz.so / libsqlite3.so.

After this PR, python.elf bundles no external Nanvix-ported libraries
at all -- every .so wrapper (_ssl, _hashlib, _ctypes, _sqlite3, zlib,
_bz2, _lzma) consumes its underlying library through DT_NEEDED. This
matches upstream cpython's default behavior on every Linux distro
(system-library detection enabled).

Changes
-------
- .nanvix/setup_local.py: add 4 *shared* entries (_bz2, _lzma, zlib,
  _sqlite3). The corresponding LIBSQLITE3_LIBS / ZLIB_LIBS /
  BZIP2_LIBS / LIBLZMA_LIBS already use -L/-l form (no change needed
  there); cpython's normal MODULE_*_LDFLAGS machinery wires them into
  each .so's link line as DT_NEEDED.

- Makefile.nanvix: drop -lsqlite3 -lz -lbz2 -llzma from python.elf's
  LIBS (those symbols come from the standalone .so files now). LIBS
  segment 2 becomes empty.

- .nanvix/runtime_sos.py: REQUIRED_RUNTIME_SOS gains libbz2.so /
  liblzma.so / libz.so / libsqlite3.so so .nanvix/test.py and
  .nanvix/package.py stage them into sysroot/lib/ at run time.

- .nanvix/z.py: _DEP_EXPECTED_LIBS for bzip2 / zlib / sqlite / xz
  extended with the .so siblings so `./z setup` validates that the
  new shared-library port builds installed correctly into
  buildroot/lib.

- .nanvix/test.py: extend CPYTHON_TEST_EXTERNAL_DEPS smoke test with
  the 4 new modules.

Runtime dependencies
--------------------
- nanvix/nanvix#2472 -- libm visibility fix (inherited from PR-A).
- nanvix/nanvix#2473 -- dlfcn init-array + DT_RUNPATH support
  (inherited from PR-B).

The 4 new .so files this PR consumes come from the Wave 6 port-repo
PRs that must land first:
- esaurez/bzip2#2 -- libbz2.so build target
- esaurez/zlib#1 -- libz.so build target
- esaurez/xz#1 -- liblzma.so build target
- esaurez/sqlite#1 -- libsqlite3.so build target

These four port-lib releases must be pinned in cpython's nanvix.toml
before this PR is mergeable upstream.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant