[build] E: Build lxml C extensions as .so#71
Open
esaurez wants to merge 1 commit into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the Nanvix port build to produce position-independent shared objects (liblxml_etree.so, liblxml_elementpath.so) alongside the existing static archives, aiming to allow consumers (e.g., Python) to dlopen() lxml and rely on transitive DT_NEEDED resolution for libxml2/libxslt.
Changes:
- Add
.sobuild targets for the lxml modules and link them as shared libraries. - Add
package/verify-packagetargets for producing and checking a release tarball. - Update
.nanvix/z.pyto include the new.soartifacts in the staged output list and clarify build help text.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
.nanvix/Makefile.nanvix |
Adds shared-library build outputs and packaging targets; modifies build/staging/clean behavior. |
.nanvix/z.py |
Updates staged output list to include .so artifacts and updates build docstrings/help text. |
Comments suppressed due to low confidence (1)
.nanvix/Makefile.nanvix:57
- With the staging/install flow,
OUT_DIR,LIB_OUT, andTEST_OUT(and related vars) are required inputs. Dropping them from_REQUIREDmeansmake allcan run with empty vars and attempt to create/copy into paths like/release/...depending on how the shell expands them. Restoring the previous required-variable list avoids unsafe default paths and produces clearer errors when invoking the Makefile directly.
# Ensure required variables are defined.
_REQUIRED := PLATFORM PROCESS_MODE MEMORY_SIZE NANVIX_HOME NANVIX_TOOLCHAIN
ifneq ($(filter-out clean,$(_NANVIX_GOALS)),)
$(foreach v,$(_REQUIRED),\
$(if $($v),,$(error Required variable $v not set))\
)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d27c752 to
84e282d
Compare
Produce position-independent liblxml_etree.so and
liblxml_elementpath.so alongside the existing static .a archives.
The .so files form an explicit DT_NEEDED chain on top of libxml2
and libxslt:
liblxml_etree.so -> DT_NEEDED libxslt.so, libxml2.so
liblxml_elementpath.so -> (leaf, no DT_NEEDED on the chain)
so a Python interpreter (or any other host executable) can dlopen
liblxml_etree.so once and have the loader resolve libxslt.so and
libxml2.so transitively. libposix/libc/libm symbols are left UND
and bound at dlopen time against the host executable's .dynsym,
matching the extension-module model used elsewhere on Nanvix.
Concretely:
* The existing static-library build (compiled with -fPIC since this
port's first revision) is preserved unchanged.
* New `.libs/liblxml_etree.so` and `.libs/liblxml_elementpath.so`
targets link the .so files manually via `-shared -fPIC -nostdlib`,
setting DT_SONAME=<name>.so and emitting the cross-library
DT_NEEDED edges by linking through `-L. -lxslt -lxml2` (resolved
against the upstream-released libxslt.so / libxml2.so).
* `test-functional` now sanity-checks each .so: presence, minimum
size, DT_SONAME, DT_NEEDED chain (including the absence of any
unintended deps), and that the lxml extension entry points
(`PyInit_etree`, `PyInit__elementpath`) appear in .dynsym.
* `package` / `verify-package` ship the static archives plus both
.so files.
Build-time dependency: this PR's shared link requires libxml2.so
and libxslt.so release artifacts to satisfy the DT_NEEDED edges.
The companion libxml2 / libxslt PRs adding those .so files should
land first; once they have released artifacts, this PR's CI can
be re-pointed at those releases in `.nanvix/nanvix.toml`.
Runtime dependency: only useful once the loader changes in
nanvix/nanvix#2473 (DT_RUNPATH + .init_array) and nanvix/nanvix#2478
(diamond DT_NEEDED handling) ship -- the lxml chain forms a
diamond at runtime via the two paths from liblxml_etree.so into
libxml2.so.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
84e282d to
f967b3e
Compare
Comment on lines
78
to
+82
| # PYTHON_OUT is derived in the Makefile as $(OUT_DIR)/release/python-packages. | ||
| python_out = out_dir() / "release" / "python-packages" | ||
| return [ | ||
| str((lib_out() / "liblxml_etree.a").relative_to(root)), | ||
| str((lib_out() / "liblxml_etree.so").relative_to(root)), |
Comment on lines
+84
to
+86
| all: $(STATICLIB_ETREE) $(STATICLIB_ELEMENTPATH) $(SHAREDLIB_ETREE) $(SHAREDLIB_ELEMENTPATH) $(TEST_ELF) | ||
|
|
||
| # Compile the Cython-generated C sources with -fPIC so they can be |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Produce position-independent
liblxml_etree.soandliblxml_elementpath.soalongside the existing static.aarchives. The new .so files form an explicitDT_NEEDEDchain on top of the libxml2 and libxslt shared-library ports:so a Python interpreter (or any other host executable) can dlopen
liblxml_etree.soonce and have the loader resolvelibxslt.soandlibxml2.sotransitively.libposix/libc/libmsymbols are left UND and bound at dlopen time against the host executable's.dynsym, matching the extension-module model used elsewhere on Nanvix.What changes
.nanvix/Makefile.nanvix-fPICsince this port's first revision) is preserved unchanged. New.libs/liblxml_etree.soand.libs/liblxml_elementpath.sotargets link the .so files manually via-shared -fPIC -nostdlib, settingDT_SONAME=<name>.soand emitting the cross-libraryDT_NEEDEDedges by linking through-L. -lxslt -lxml2(resolved against the upstream-releasedlibxslt.so/libxml2.so).test-functionalnow sanity-checks each .so: presence, minimum size,DT_SONAME,DT_NEEDEDchain (including the absence of any unintended deps), and that the lxml extension entry points (PyInit_etree,PyInit__elementpath) appear in.dynsym.package/verify-packageship the static archives plus both .so files..nanvix/z.py.sofiles to_staged_output_files()soreleasepackages them.Build-time dependency
This PR's shared link requires
libxml2.soandlibxslt.sorelease artifacts (from the companion libxml2 and libxslt PRs adding shared-library support) to satisfy theDT_NEEDEDedges on the link line. The companion PRs should land first; once they have released artifacts, this PR's CI can be re-pointed at those releases in.nanvix/nanvix.toml.Runtime dependencies
Only useful once these loader changes ship:
DT_RUNPATH+.init_arrayinvocation.DT_NEEDEDhandling. The lxml chain forms a diamond at runtime via the two paths fromliblxml_etree.sointolibxml2.so.