Skip to content

rpm_pfg: allow multiple tree artifacts at the same destination#1053

Merged
aiuto merged 3 commits into
bazelbuild:mainfrom
chouquette:rpm-allow-shared-tree-dest
Apr 22, 2026
Merged

rpm_pfg: allow multiple tree artifacts at the same destination#1053
aiuto merged 3 commits into
bazelbuild:mainfrom
chouquette:rpm-allow-shared-tree-dest

Conversation

@chouquette
Copy link
Copy Markdown
Contributor

@chouquette chouquette commented Apr 16, 2026

When two pkg_files targets both map a configure_make tree artifact (e.g. an include/ directory) to the same destination prefix, pkg_rpm was raising a hard error even though the file contents don't overlap. The duplicate-destination check was firing unconditionally before the src.is_directory branch, so two packages installing headers to embedded/include/ (from different subdirectories) would conflict.

Move the dest_check_map conflict check inside the non-directory branch so that tree artifacts sharing a destination are treated as a merge rather than a conflict. Actual file-level collisions within those trees will be caught at install time by the underlying tooling.

@chouquette chouquette mentioned this pull request Apr 16, 2026
Merged
cgrindel
cgrindel reviewed Apr 19, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@cgrindel cgrindel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable. Is there a way to add a test for this?

@chouquette
rpm_pfg: allow multiple tree artifacts at the same destination
17d7ec0 
When two pkg_files targets both map a configure_make tree artifact
(e.g. an include/ directory) to the same destination prefix, pkg_rpm
was raising a hard error even though the file contents don't overlap.
The duplicate-destination check was firing unconditionally before the
src.is_directory branch, so two packages installing headers to
embedded/include/ (from different subdirectories) would conflict.

Move the dest_check_map conflict check inside the non-directory branch
so that tree artifacts sharing a destination are treated as a merge
rather than a conflict. Actual file-level collisions within those trees
will be caught at install time by the underlying tooling.
@chouquette
pkg_files: allow multiple tree artifacts at the same destination
426b64c 
Two TreeArtifacts mapping to the same destination directory should be
permitted; their contents are merged by the installer. File-level
conflicts are still caught naturally when the same destination file
path is claimed by two different sources.

This is the general-purpose counterpart of the rpm_pfg fix in 8f67a8a.
@chouquette
tests: add a couple tests for the previous commit
0b5c59d
@chouquette chouquette force-pushed the rpm-allow-shared-tree-dest branch from 011cc9c to 0b5c59d Compare April 22, 2026 08:15
@aiuto aiuto added the rpm label Apr 22, 2026
@aiuto aiuto merged commit 401969d into bazelbuild:main Apr 22, 2026
6 checks passed
@chouquette chouquette mentioned this pull request Apr 23, 2026
Merged
gh-worker-dd-mergequeue-cf854d Bot pushed a commit to DataDog/datadog-agent that referenced this pull request Apr 23, 2026
@chouquette
bump rules_pkg commit to the most recent commit (#49755)
1d3b67a 
### What does this PR do?

Update rules_pkg to the latest commit on main

### Motivation

This fixes potential conflicts when 2 targets attempt to create a similar folder hierarchy, even if the final files aren't overlapping
See related PR: bazelbuild/rules_pkg#1053

### Describe how you validated your changes

### Additional Notes


Co-authored-by: hugo.beauzee <hugo.beauzee@datadoghq.com>
gh-worker-dd-mergequeue-cf854d Bot pushed a commit to DataDog/datadog-agent that referenced this pull request May 11, 2026
@chouquette
bazel: migrate curl & openssl to dd_cc_packaged & dd_collect_dependen…
ca6ff35 
…cies (#49403)

### What does this PR do?

Continue the migration toward installing dependencies with dd_collect_dependencies
This PR mostly migrates OpenSSL, Python & rtloader.

It also contains some changes to our rules:
* ` foreign_cc_shared_wrapper`: 
    * stop re-exporting the wrapped target's providers
    * add lib_filter to allow producing multiple wrappers objects for multiple libraries from a single configure_make target
    * return DefaultInfo so the wrapper can be used as a data dependency                                                    
* `dd_cc_packaged`: add `installed_executables` attribute to rpath-patch and
  install executables/shared modules (engines, ossl-modules, …) alongside the library                                                                 
* `rewrite_rpath`: extract `patchelf_file_action` / `otool_file_action` & introduce `patchelf_dir_action` / `otool_dir_action` as reusable helpers. We now can rpath-patch entire output directories (useful for python modules & openssl engines)

### Motivation

Migrating away from omnibus.

### Describe how you validated your changes

Local `bazel test //...` and CI

### Additional Notes

~Based on top of #49386 (merged)
~Based on top of #49500 (merged)
~The rules_pkg change is opened upstream at bazelbuild/rules_pkg#1053 (change merged & `rules_pkg` commit bumped in `main`)

The preliminary changes to our tooling (`foreign_cc_shared_wrapper`, `dd_cc_packaged`'s `installed_executable` & `rewrite_rpath` refactoring can each be split out in their own PR if it simplifies the review)

The heroku size increase is coming from license files that weren't shipped in the package before:
```
 - LICENSES/krb5-NOTICE — 64 KB 
  - LICENSES/rpm-COPYING — 44 KB 
  - LICENSES/gstatus-LICENSE — 35 KB 
  - LICENSES/acl-COPYING.LGPL — 26 KB
  - LICENSES/attr-COPYING.LGPL — 26 KB
  - LICENSES/gcrypt-COPYING.LIB — 26 KB
  - LICENSES/systemd-LICENSE.LGPL2.1 — 26 KB
  - LICENSES/libsepol-LICENSE — 26 KB 
  - LICENSES/freetds-COPYING.LIB — 25 KB
  - LICENSES/unixodbc-COPYING — 24 KB 
  - LICENSES/cacerts-MPL-2.0.txt — 16 KB 
  - LICENSES/msodbcsql18-LICENSE.txt — 12 KB
  - LICENSES/xmlsec-Copyright — 7 KB 
  - LICENSES/jmxfetch-LICENSE — 2 KB
  - LICENSES/dbus-COPYING — 1 KB
  - LICENSES/popt-COPYING — 1 KB
  - LICENSES/attr-LICENSE — 1 KB
  - sources/acl/offer.txt — 128 B
  - sources/attr/offer.txt — 129 B
  - sources/freetds/offer.txt — 133 B
  - sources/libsepol/offer.txt — 131 B
  - sources/systemd/offer.txt — 130 B
```

Co-authored-by: hugo.beauzee <hugo.beauzee@datadoghq.com>
chouetz pushed a commit to DataDog/datadog-agent that referenced this pull request May 13, 2026
@chouquette @chouetz
bazel: migrate curl & openssl to dd_cc_packaged & dd_collect_dependen…
e0a286c 
…cies (#49403)

### What does this PR do?

Continue the migration toward installing dependencies with dd_collect_dependencies
This PR mostly migrates OpenSSL, Python & rtloader.

It also contains some changes to our rules:
* ` foreign_cc_shared_wrapper`: 
    * stop re-exporting the wrapped target's providers
    * add lib_filter to allow producing multiple wrappers objects for multiple libraries from a single configure_make target
    * return DefaultInfo so the wrapper can be used as a data dependency                                                    
* `dd_cc_packaged`: add `installed_executables` attribute to rpath-patch and
  install executables/shared modules (engines, ossl-modules, …) alongside the library                                                                 
* `rewrite_rpath`: extract `patchelf_file_action` / `otool_file_action` & introduce `patchelf_dir_action` / `otool_dir_action` as reusable helpers. We now can rpath-patch entire output directories (useful for python modules & openssl engines)

### Motivation

Migrating away from omnibus.

### Describe how you validated your changes

Local `bazel test //...` and CI

### Additional Notes

~Based on top of #49386 (merged)
~Based on top of #49500 (merged)
~The rules_pkg change is opened upstream at bazelbuild/rules_pkg#1053 (change merged & `rules_pkg` commit bumped in `main`)

The preliminary changes to our tooling (`foreign_cc_shared_wrapper`, `dd_cc_packaged`'s `installed_executable` & `rewrite_rpath` refactoring can each be split out in their own PR if it simplifies the review)

The heroku size increase is coming from license files that weren't shipped in the package before:
```
 - LICENSES/krb5-NOTICE — 64 KB 
  - LICENSES/rpm-COPYING — 44 KB 
  - LICENSES/gstatus-LICENSE — 35 KB 
  - LICENSES/acl-COPYING.LGPL — 26 KB
  - LICENSES/attr-COPYING.LGPL — 26 KB
  - LICENSES/gcrypt-COPYING.LIB — 26 KB
  - LICENSES/systemd-LICENSE.LGPL2.1 — 26 KB
  - LICENSES/libsepol-LICENSE — 26 KB 
  - LICENSES/freetds-COPYING.LIB — 25 KB
  - LICENSES/unixodbc-COPYING — 24 KB 
  - LICENSES/cacerts-MPL-2.0.txt — 16 KB 
  - LICENSES/msodbcsql18-LICENSE.txt — 12 KB
  - LICENSES/xmlsec-Copyright — 7 KB 
  - LICENSES/jmxfetch-LICENSE — 2 KB
  - LICENSES/dbus-COPYING — 1 KB
  - LICENSES/popt-COPYING — 1 KB
  - LICENSES/attr-LICENSE — 1 KB
  - sources/acl/offer.txt — 128 B
  - sources/attr/offer.txt — 129 B
  - sources/freetds/offer.txt — 133 B
  - sources/libsepol/offer.txt — 131 B
  - sources/systemd/offer.txt — 130 B
```

Co-authored-by: hugo.beauzee <hugo.beauzee@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cgrindel cgrindel cgrindel left review comments

@aiuto aiuto aiuto approved these changes

Assignees

No one assigned

Labels

rpm

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@chouquette @cgrindel @aiuto