[nanvix] E: Build _ssl/_hashlib/_ctypes as .so#738
Open
esaurez wants to merge 1 commit into
Open
Conversation
Converts the three stdlib extensions whose external dependencies are Nanvix-ported libraries that already ship as .so (libssl/libcrypto from openssl, libffi) from *static* (linked into python.elf) to *shared* (lib/python3.12/lib-dynload/<name>.cpython-312.so), each emitting DT_NEEDED for the corresponding sysroot library: _ctypes.cpython-312.so -> libffi.so _ssl.cpython-312.so -> libssl.so -> libcrypto.so _hashlib.cpython-312.so -> libcrypto.so (shared with _ssl, deduped) This is exactly how upstream cpython builds these modules on a Linux distro with --with-system-* detection enabled (the default). The Nanvix dynamic loader walks the DT_NEEDED chain at dlopen time (the openssl chain forms a diamond at libcrypto.so) and UND symbols in each .so bind to python.elf .dynsym. _ssl and _hashlib previously embedded their own copies of libcrypto (~5.5 MB each); routing both through the single libcrypto.so ends the OpenSSL init-order bug where the two copies maintained separate provider state and the first openssl_sha256() call failed. python.elf LIBS drops -lssl -lcrypto -lffi. _bz2 / _lzma / zlib / _sqlite3 are intentionally NOT moved here: the Nanvix port repos for libbz2 / liblzma / libz / libsqlite3 do not yet ship .so builds, so those four extensions stay statically built into python.elf (cpython upstream default) until the follow-up PR that lands alongside the Wave 6 port-repo .so PRs. Infrastructure -------------- - .nanvix/runtime_sos.py (new): REQUIRED_RUNTIME_SOS single source of truth (libffi.so, libcrypto.so, libssl.so) + stage_runtime_sos(), consumed by .nanvix/test.py and .nanvix/package.py to copy the .so files from the buildroot into the test / release sysroot. - .nanvix/setup_local.py: _ssl / _hashlib / _ctypes shared entries. - .nanvix/test.py: CPYTHON_TEST_EXTERNAL_DEPS smoke checks. - .nanvix/z.py: _DEP_EXPECTED_LIBS validates libffi.so + libssl.so / libcrypto.so installed into the buildroot at `./z setup` time. - Makefile.nanvix: LIBS drops -lssl -lcrypto -lffi. Runtime dependencies (already merged upstream) ---------------------------------------------- - nanvix/nanvix#2473 -- dlfcn init-array + DT_RUNPATH support. - nanvix/nanvix#2478 -- diamond DT_NEEDED handling (openssl chain). - nanvix/nanvix#2481 -- pthread_once (critical: OpenSSL routes every internal one-time init through pthread_once). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Converts the three stdlib extensions whose external dependencies are Nanvix-ported libraries that already ship as
.so(libssl/libcryptofrom openssl,libffi) from*static*(linked intopython.elf) to*shared*(lib/python3.12/lib-dynload/<name>.cpython-312.so), each emittingDT_NEEDEDfor the corresponding sysroot library.This is exactly how upstream CPython builds these modules on a Linux distro with
--with-system-*detection enabled (the default).DT_NEEDED chains
_ctypes.cpython-312.so→DT_NEEDED libffi.so_ssl.cpython-312.so→DT_NEEDED libssl.so→DT_NEEDED libcrypto.so_hashlib.cpython-312.so→DT_NEEDED libcrypto.soThe Nanvix dynamic loader walks the chain at
dlopen()time (the openssl chain forms a diamond atlibcrypto.so), and UND symbols in each.sobind topython.elf .dynsym. Same flow Linux distros use._ssland_hashlibpreviously embedded their own copies oflibcrypto(~5.5 MB each); routing both through the singlelibcrypto.soends the OpenSSL init-order bug where the two copies maintained separate provider state and the firstopenssl_sha256()call failed.Scope
python.elfLIBS drops-lssl -lcrypto -lffi._bz2/_lzma/zlib/_sqlite3are intentionally not moved here — the Nanvix port repos forlibbz2/liblzma/libz/libsqlite3do not yet ship.sobuilds, so those four extensions stay statically built intopython.elf(CPython upstream default) until those port repos ship.sobuilds and a follow-up PR switches the four extensions toDT_NEEDED.lxml is not part of this PR. lxml is a third-party package, not a CPython stdlib module, and the way it is currently integrated on Nanvix (a flat-name built-in shim) diverges from how upstream CPython loads third-party extensions. That divergence is addressed separately — this PR stays focused on the stdlib modules that match upstream's system-library
.soflow.Mechanics
.nanvix/runtime_sos.py(new)REQUIRED_RUNTIME_SOSsingle source of truth (libffi.so,libcrypto.so,libssl.so) +stage_runtime_sos(), consumed by.nanvix/test.pyand.nanvix/package.pyto copy the.sofiles from the buildroot into the test / release sysroot..nanvix/setup_local.py_ssl/_hashlib/_ctypesshared entries..nanvix/test.pyCPYTHON_TEST_EXTERNAL_DEPSsmoke checks (import+ trivial attribute access)..nanvix/z.py_DEP_EXPECTED_LIBSvalidateslibffi.so+libssl.so/libcrypto.soinstalled into the buildroot at./z setuptime.Makefile.nanvix-lssl -lcrypto -lffi.Dependencies
Base branch:
feat/wave5-pr-a-stdlib-so(#732).Port-repo PRs that produce the required
.sofiles (already filed upstream):.solibffi.solibcrypto.so,libssl.soRuntime dependencies (already merged upstream):
dlfcninit-array + DT_RUNPATH support.libcrypto.soboth vialibssl.soand directly from_hashlib).pthread_once(critical: OpenSSL routes every internal one-time init throughpthread_once; without it everyEVP_*call fails).