[nanvix] E: Build bz2/lzma/zlib/sqlite3 as DT_NEEDED .so chains#739
Open
esaurez wants to merge 1 commit into
Open
[nanvix] E: Build bz2/lzma/zlib/sqlite3 as DT_NEEDED .so chains#739esaurez wants to merge 1 commit into
esaurez wants to merge 1 commit into
Conversation
Completes the external-library unbundling started in the parent PR by moving the remaining four stdlib extensions (_bz2, _lzma, zlib, _sqlite3) from being statically linked into python.elf to runtime- loaded .so files that emit DT_NEEDED for libbz2.so / liblzma.so / libz.so / libsqlite3.so. After this PR, python.elf bundles no external Nanvix-ported libraries at all -- every .so wrapper (_ssl, _hashlib, _ctypes, _sqlite3, zlib, _bz2, _lzma) consumes its underlying library through DT_NEEDED. This matches upstream cpython's default behavior on every Linux distro (system-library detection enabled). Changes ------- - .nanvix/setup_local.py: add 4 *shared* entries (_bz2, _lzma, zlib, _sqlite3). The corresponding LIBSQLITE3_LIBS / ZLIB_LIBS / BZIP2_LIBS / LIBLZMA_LIBS already use -L/-l form (no change needed there); cpython's normal MODULE_*_LDFLAGS machinery wires them into each .so's link line as DT_NEEDED. - Makefile.nanvix: drop -lsqlite3 -lz -lbz2 -llzma from python.elf's LIBS (those symbols come from the standalone .so files now). LIBS segment 2 becomes empty. - .nanvix/runtime_sos.py: REQUIRED_RUNTIME_SOS gains libbz2.so / liblzma.so / libz.so / libsqlite3.so so .nanvix/test.py and .nanvix/package.py stage them into sysroot/lib/ at run time. - .nanvix/z.py: _DEP_EXPECTED_LIBS for bzip2 / zlib / sqlite / xz extended with the .so siblings so `./z setup` validates that the new shared-library port builds installed correctly into buildroot/lib. - .nanvix/test.py: extend CPYTHON_TEST_EXTERNAL_DEPS smoke test with the 4 new modules. Runtime dependencies -------------------- - nanvix/nanvix#2472 -- libm visibility fix (inherited from PR-A). - nanvix/nanvix#2473 -- dlfcn init-array + DT_RUNPATH support (inherited from PR-B). The 4 new .so files this PR consumes come from the Wave 6 port-repo PRs that must land first: - esaurez/bzip2#2 -- libbz2.so build target - esaurez/zlib#1 -- libz.so build target - esaurez/xz#1 -- liblzma.so build target - esaurez/sqlite#1 -- libsqlite3.so build target These four port-lib releases must be pinned in cpython's nanvix.toml before this PR is mergeable upstream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the external-library unbundling started in #738 by moving the remaining four stdlib extensions (
_bz2,_lzma,zlib,_sqlite3) from being statically linked intopython.elfto runtime-loaded.sofiles emittingDT_NEEDEDforlibbz2.so/liblzma.so/libz.so/libsqlite3.so.After this PR,
python.elfbundles no external Nanvix-ported libraries at all — every.sowrapper (_ssl,_hashlib,_ctypes,_sqlite3,zlib,_bz2,_lzma) consumes its underlying library through DT_NEEDED. This matches upstream cpython's default behavior on every Linux distro (system-library detection enabled).Why
The previous PR (#738) took the libffi / openssl / lxml consumers to DT_NEEDED
.sochains but left these four extensions statically linked intopython.elfbecause the Nanvix port repos forbzip2/zlib/xz/sqlitedid not yet ship.sobuilds. The four port-repo PRs listed under Dependencies below add those.sobuilds, which finally lets this PR finish the migration.What changed
4 stdlib modules migrated from
*static*(linked intopython.elf) to*shared*(loaded via dlopen). The correspondingLIBSQLITE3_LIBS/ZLIB_LIBS/BZIP2_LIBS/LIBLZMA_LIBSenv vars already use-L/-lform in #738 (no change needed there); cpython's normalMODULE_*_LDFLAGSmachinery wires them into each.so's link line as DT_NEEDED.The Nanvix dynamic loader walks each DT_NEEDED chain at
dlopen()time, and UND symbols in any of those.sofiles bind topython.elf .dynsym.Mechanics
.nanvix/setup_local.py*shared*entries (_bz2,_lzma,zlib,_sqlite3).Makefile.nanvix-lsqlite3 -lz -lbz2 -llzmafrompython.elfLIBS; the-Wl,--start-group ... -Wl,--end-groupblock is gone. Per-module*_LIBSenv vars unchanged (already in-L/-lform from #738)..nanvix/runtime_sos.pyREQUIRED_RUNTIME_SOSgainslibbz2.so/liblzma.so/libz.so/libsqlite3.soso.nanvix/test.pyand.nanvix/package.pystage them intosysroot/lib/at run time..nanvix/z.py_DEP_EXPECTED_LIBSforbzip2,zlib,sqlite,xzextended with the.sosiblings so./z setupvalidates that the new shared-library port builds installed correctly intobuildroot/lib..nanvix/test.pyCPYTHON_TEST_EXTERNAL_DEPSsmoke test with the 4 new modules.Dependencies
Base branch: #738.
Runtime dependencies — the four port-repo PRs that produce the
.sofiles this PR consumes:libbz2.sobuild target.libz.sobuild target.liblzma.sobuild target.libsqlite3.sobuild target.These four PRs must land and their port-lib releases must be pinned in cpython's
nanvix.tomlbefore this PR is mergeable upstream.Also inherits the runtime deps already required by #738:
dlfcninit-array + DT_RUNPATH support (already merged upstream).pthread_oncefix described in [nanvix] E: Build _ssl/_hashlib/_ctypes as .so #738 (in-flight innanvix/nanvix).Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com