Skip to content

[nanvix] E: Build _ssl/_hashlib/_ctypes as .so#738

Open
esaurez wants to merge 1 commit into
feat/wave5-pr-a-stdlib-sofrom
feat/wave5-pr-b-unbundle-externals
Open

[nanvix] E: Build _ssl/_hashlib/_ctypes as .so#738
esaurez wants to merge 1 commit into
feat/wave5-pr-a-stdlib-sofrom
feat/wave5-pr-b-unbundle-externals

Conversation

@esaurez

@esaurez esaurez commented Jun 16, 2026

Copy link
Copy Markdown

Summary

Converts the three stdlib extensions whose external dependencies are Nanvix-ported libraries that already ship as .so (libssl/libcrypto from openssl, libffi) from *static* (linked into python.elf) to *shared* (lib/python3.12/lib-dynload/<name>.cpython-312.so), each emitting DT_NEEDED for the corresponding sysroot library.

This is exactly how upstream CPython builds these modules on a Linux distro with --with-system-* detection enabled (the default).

DT_NEEDED chains

  • _ctypes.cpython-312.soDT_NEEDED libffi.so
  • _ssl.cpython-312.soDT_NEEDED libssl.soDT_NEEDED libcrypto.so
  • _hashlib.cpython-312.soDT_NEEDED libcrypto.so

The Nanvix dynamic loader walks the chain at dlopen() time (the openssl chain forms a diamond at libcrypto.so), and UND symbols in each .so bind to python.elf .dynsym. Same flow Linux distros use.

_ssl and _hashlib previously embedded their own copies of libcrypto (~5.5 MB each); routing both through the single libcrypto.so ends the OpenSSL init-order bug where the two copies maintained separate provider state and the first openssl_sha256() call failed.

Scope

python.elf LIBS drops -lssl -lcrypto -lffi. _bz2 / _lzma / zlib / _sqlite3 are intentionally not moved here — the Nanvix port repos for libbz2 / liblzma / libz / libsqlite3 do not yet ship .so builds, so those four extensions stay statically built into python.elf (CPython upstream default) until those port repos ship .so builds and a follow-up PR switches the four extensions to DT_NEEDED.

lxml is not part of this PR. lxml is a third-party package, not a CPython stdlib module, and the way it is currently integrated on Nanvix (a flat-name built-in shim) diverges from how upstream CPython loads third-party extensions. That divergence is addressed separately — this PR stays focused on the stdlib modules that match upstream's system-library .so flow.

Mechanics

File Change
.nanvix/runtime_sos.py (new) REQUIRED_RUNTIME_SOS single source of truth (libffi.so, libcrypto.so, libssl.so) + stage_runtime_sos(), consumed by .nanvix/test.py and .nanvix/package.py to copy the .so files from the buildroot into the test / release sysroot.
.nanvix/setup_local.py _ssl / _hashlib / _ctypes shared entries.
.nanvix/test.py CPYTHON_TEST_EXTERNAL_DEPS smoke checks (import + trivial attribute access).
.nanvix/z.py _DEP_EXPECTED_LIBS validates libffi.so + libssl.so / libcrypto.so installed into the buildroot at ./z setup time.
Makefile.nanvix LIBS drops -lssl -lcrypto -lffi.

Dependencies

Base branch: feat/wave5-pr-a-stdlib-so (#732).

Port-repo PRs that produce the required .so files (already filed upstream):

Required .so Produced by
libffi.so nanvix/libffi#235
libcrypto.so, libssl.so nanvix/openssl#252

Runtime dependencies (already merged upstream):

Converts the three stdlib extensions whose external dependencies are
Nanvix-ported libraries that already ship as .so (libssl/libcrypto
from openssl, libffi) from *static* (linked into python.elf) to
*shared* (lib/python3.12/lib-dynload/<name>.cpython-312.so), each
emitting DT_NEEDED for the corresponding sysroot library:

  _ctypes.cpython-312.so   -> libffi.so
  _ssl.cpython-312.so      -> libssl.so -> libcrypto.so
  _hashlib.cpython-312.so  -> libcrypto.so  (shared with _ssl, deduped)

This is exactly how upstream cpython builds these modules on a Linux
distro with --with-system-* detection enabled (the default). The
Nanvix dynamic loader walks the DT_NEEDED chain at dlopen time (the
openssl chain forms a diamond at libcrypto.so) and UND symbols in
each .so bind to python.elf .dynsym.

_ssl and _hashlib previously embedded their own copies of libcrypto
(~5.5 MB each); routing both through the single libcrypto.so ends the
OpenSSL init-order bug where the two copies maintained separate
provider state and the first openssl_sha256() call failed.

python.elf LIBS drops -lssl -lcrypto -lffi. _bz2 / _lzma / zlib /
_sqlite3 are intentionally NOT moved here: the Nanvix port repos for
libbz2 / liblzma / libz / libsqlite3 do not yet ship .so builds, so
those four extensions stay statically built into python.elf (cpython
upstream default) until the follow-up PR that lands alongside the
Wave 6 port-repo .so PRs.

Infrastructure
--------------
- .nanvix/runtime_sos.py (new): REQUIRED_RUNTIME_SOS single source of
  truth (libffi.so, libcrypto.so, libssl.so) + stage_runtime_sos(),
  consumed by .nanvix/test.py and .nanvix/package.py to copy the .so
  files from the buildroot into the test / release sysroot.
- .nanvix/setup_local.py: _ssl / _hashlib / _ctypes shared entries.
- .nanvix/test.py: CPYTHON_TEST_EXTERNAL_DEPS smoke checks.
- .nanvix/z.py: _DEP_EXPECTED_LIBS validates libffi.so + libssl.so /
  libcrypto.so installed into the buildroot at `./z setup` time.
- Makefile.nanvix: LIBS drops -lssl -lcrypto -lffi.

Runtime dependencies (already merged upstream)
----------------------------------------------
- nanvix/nanvix#2473 -- dlfcn init-array + DT_RUNPATH support.
- nanvix/nanvix#2478 -- diamond DT_NEEDED handling (openssl chain).
- nanvix/nanvix#2481 -- pthread_once (critical: OpenSSL routes every
  internal one-time init through pthread_once).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant