instructions: document AZL component-modification workflow conventions#17352
Draft
PawelWMS wants to merge 1 commit into
Draft
instructions: document AZL component-modification workflow conventions#17352PawelWMS wants to merge 1 commit into
PawelWMS wants to merge 1 commit into
Conversation
Captures the conventions developed during the multi-day session of
remediating package-signing/scan failures at the component level
(no allow-listing) and folds them into permanent Copilot guidance.
No new tooling -- this is purely instructional / documentation
content to make future agent sessions discover and follow the
patterns we now use across the distro.
Changes
-------
1. `.github/instructions/comp-toml.instructions.md` (+69 lines).
New "replace-upstream source override" section documenting the
single-step pattern for serving a locally-modified upstream
tarball under the same filename: one
`[[components.<name>.source-files]]` block with
`replace-upstream = true` and a `replace-reason = "..."` string.
No separate `file-remove` overlay against `sources` is needed;
`azldev`'s render step emits an audit WARN log naming the
override and the from/to SHA-512 pair.
Covers: field semantics (`filename`, `hash`, `hash-type`,
`replace-upstream`, `replace-reason`, `origin.uri`); the
lookaside URL pattern; the requirement that `hash` and the
`$hash` segment of `origin.uri` stay in sync; the
`replace-reason` style rules (single TOML string, no
triple-quoted strings, self-contained motivation); and a
"Public-content hygiene" subsection establishing that
motivations be described technically and neutrally (no specific
internal infrastructure names).
Also documents the `components.toml` file-organization rule:
when moving a component to a dedicated `*.comp.toml`, delete
the inline entry; when removing a component, delete the inline
entry. No tombstone comments.
2. `.github/skills/skill-modify-source/SKILL.md` (new, 267 lines).
New skill documenting the canonical `modify_source.sh` pattern
for byte-deterministic Source0 repacks. The lookaside URL
embeds the SHA-512 of the served file, so the repack MUST be
byte-deterministic: same upstream input ⇒ byte-identical
output ⇒ identical SHA-512, across machines and re-runs.
Sections:
(a) Anatomy of `modify_source.sh` (scratch dir under
`base/build/work/scratch/<name>/`, `BASH_SOURCE[0]`
root resolution, `set -euo pipefail`).
(b) The byte-deterministic repack contract -- required `tar`
flags (`--sort=name`, `--owner=0 --group=0`,
`--numeric-owner`, `--mtime='@<epoch>'`, `--format=gnu`,
`LC_ALL=C`) and `xz` flags (`-T 1`, `-9`), plus the
forbidden set.
(c) Mandatory upstream SHA-512 verification before extract.
(d) Strip-list vs keep-list strategies with list-discipline
rules.
(e) Output-and-reporting contract (path, hash, ready-to-paste
`az storage blob upload` command, comp.toml update
reminder).
(f) Determinism self-test recipe (two consecutive runs must
produce byte-identical output).
(g) Anti-patterns (`mktemp` for output filename, `$$` / `$RANDOM`
/ `$(date)` intermediates, `gzip` without `-n`, multi-
threaded `xz` without `--block-size`, `find -newer`, etc.).
(h) Canonical ~40-line bash skeleton showing the whole pattern
end-to-end.
3. `.github/skills/skill-signing-failure-remediation/SKILL.md`
(new, 98 lines). New skill documenting the decision tree for
handling pipeline scan failures at the component level:
(a) No reverse dependencies → drop the component outright
(separate PR).
(b) Offending artefact lives in a test-only / optional
subpackage → use `spec-remove-subpackage` (or
`build.without` if a clean bcond exists).
(c) Reverse deps require the component to keep building →
strip subtrees from Source0 via the `replace-upstream`
pattern with a `modify_source.sh` script
(see `skill-modify-source`).
Always run a reverse-dependency scan before choosing (a) or (b).
4. `.github/copilot-instructions.md` (+3 lines). Three new
repository-wide rules at the bottom of "Repository Hygiene
Rules":
6. Commit signing -- every commit MUST be GPG-signed;
`git log -1 --show-signature` must show "Good signature".
7. Public-content hygiene -- describe motivations in overlay
descriptions, replace-reason strings, comp.toml comments,
spec files, modify_source.sh echo lines, commit messages,
and PR descriptions *technically and neutrally*. Use
neutral phrasing such as "automated package-signing
pipeline", "FS-aware deep scanner", "automated malware
scan".
8. Component-modification order of operations -- canonical
sequence is `edit comp.toml → azldev comp update -p <name>
→ commit (toml + lock) → azldev comp render -p <name> →
amend (rendered spec)`. Keeps the lock fingerprint,
%changelog, and Release: in sync within a single commit.
5. `.github/skills/skill-update-component/SKILL.md` (+20/-4 lines).
Clarified the commit ordering with a worked example. The
canonical sequence is: edit `comp.toml`, `azldev comp update -p
<name>` to refresh the lock, commit the comp.toml + lock,
`azldev comp render -p <name>`, amend in the rendered spec.
Spelled out *why* the second render-and-amend is needed
(`%changelog` / `Release:` are derived from `git log`, not the
working tree).
6. `AGENTS.md` (+2 lines). Added two rows to the skill index:
| Byte-deterministic Source0 repack (`modify_source.sh` +
`replace-upstream`) | `skill-modify-source` |
| Remediate component-level signing/scan failures |
`skill-signing-failure-remediation` |
7. `base/comps/AGENTS.md` (+4 lines). Mirrored the
`components.toml` hygiene rule at the directory-level guide:
when moving a component to a dedicated file, delete the inline
entry (no tombstone comment); when removing a component,
delete the inline entry.
8. `.github/copilot-instructions.md` (+1 line, follow-up). Added
rule 11 to "Repository Hygiene Rules": fix render / lock
drift locally by re-running `azldev comp render -p <name>` /
`azldev comp update -p <name>` and amending, never by
`gh run download`-ing the `rendered-specs-patch` /
`locks-patch` CI artifact and `git apply`-ing it. The
artifact is a reviewer convenience; using it as a fix path
skips local reproduction and masks any local/CI environment
skew.
9. `.github/skills/skill-update-component/SKILL.md` (+2 lines,
follow-up). Added a "Drift-fix discipline" paragraph under the
existing "CI gotcha" section, cross-referencing rule 11.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR documents Azure Linux component-modification workflow conventions (source overrides, signing/scan remediation, and update/render/commit ordering) to make them discoverable and repeatable for future contributors and Copilot/agent sessions.
Changes:
- Documented the
replace-upstreamsource-override pattern and related hygiene/workflow guidance in component TOML instructions. - Added two new skills (
skill-modify-source,skill-signing-failure-remediation) and expanded troubleshooting guidance inskill-fix-overlay. - Clarified component update/render commit ordering and added additional repository workflow rules in Copilot instructions.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
base/comps/AGENTS.md |
Adds components.toml hygiene rules and indexes the new skills for component work. |
AGENTS.md |
Adds the two new skills to the repo-wide skill index. |
.github/skills/skill-update-component/SKILL.md |
Clarifies the commit/update/render/amend order and adds local drift-fix discipline guidance. |
.github/skills/skill-signing-failure-remediation/SKILL.md |
New decision-tree skill for remediation paths (drop component/subpackage vs strip Source0). |
.github/skills/skill-modify-source/SKILL.md |
New skill documenting a deterministic modify_source.sh repack pattern for replace-upstream. |
.github/skills/skill-fix-overlay/SKILL.md |
Adds guidance on section scoping pitfalls and %if/%endif hazards when removing sections/subpackages. |
.github/instructions/comp-toml.instructions.md |
Documents replace-upstream, public-content hygiene, file-organization rules, and build-flag override guidance. |
.github/copilot-instructions.md |
Adds azldev install/update guidance plus commit signing, hygiene, and component-modification ordering rules. |
Comments suppressed due to low confidence (2)
.github/skills/skill-signing-failure-remediation/SKILL.md:72
- The reverse-dependency scan guidance mentions
base/comps/*/components.toml, but this repo’s component list isbase/comps/components.toml(no per-subdircomponents.toml). Pointing at a nonexistent path makes the checklist harder to follow and can lead to incomplete scans.
- `Requires:` / `BuildRequires:` references in other components' specs or comp.toml overlays.
- Image manifest inclusions (`base/images/**`).
- References in `base/comps/*/components.toml` or any other comp.toml as a dependency.
.github/instructions/comp-toml.instructions.md:164
- The new
replace-upstream/replace-reasonkeys shown here aren’t currently part of the authoritative schema (external/schemas/azldev.schema.json→SourceFileReference, which hasadditionalProperties: false). That means schema-based validation/IDE tooling will flag this example (and any real usage) as invalid unless the schema is updated or the docs call out the mismatch explicitly.
filename = "examplepkg-1.2.3.tar.xz"
hash = "<sha512 of LOCALLY-MODIFIED tarball>"
hash-type = "SHA512"
replace-upstream = true
replace-reason = "Upstream tarball ships a vendored copy of a third-party library plus a 'tests/network' tree that trip an automated package-signing pipeline's deep scanner. Repacked under the same filename with those subtrees stripped via base/comps/examplepkg/modify_source.sh; the surviving bits are byte-identical to upstream so the build is unaffected."
|
|
||
| ```bash | ||
| # Replace <pkg> with the component name and <sub> with the sub-package suffix (or just <pkg> alone). | ||
| grep -rn '<pkg>-<sub>\|<pkg>\b' base/comps/ specs/ base/images/ 2>/dev/null | head -50 |
| - **Dedicated** (`<name>/<name>.comp.toml`): when overlays, build config, or local spec are needed | ||
| - Rule of thumb: if it's more than `[components.<name>]`, give it a dedicated file | ||
| - `components.toml` has `includes = ["**/*.comp.toml"]` — dedicated files are picked up automatically | ||
| - **When moving a component out of inline `components.toml` into a dedicated `<name>.comp.toml` file, DELETE the inline entry.** Don't leave a "moved to X" pointer comment. Discovery is automatic via the `**/*.comp.toml` include glob. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR captures the conventions developed during a long-running multi-day session of remediating package-signing/scan failures at the component level (no allow-listing). The goal is to make those conventions discoverable to future Copilot sessions and agents, not to introduce new tooling.
What this PR adds
1.
replace-upstreamsource-override pattern (comp-toml.instructions.md)Documents the single-step pattern for serving a locally-modified upstream tarball under the same filename: one
[[components.<name>.source-files]]block withreplace-upstream = trueand areplace-reason = "..."string. No separatefile-removeoverlay is needed;azldev's render step emits an audit WARN log naming the override and the from/to SHA-512 pair.Covers field semantics, the lookaside URL pattern, the requirement that
hashand the$hashsegment oforigin.uristay in sync,replace-reasonstyle rules, and the file-organization rules (delete inlinecomponents.tomlentries when moving to a dedicated*.comp.toml; no tombstone comments).2.
skill-modify-sourceskillNew skill at
.github/skills/skill-modify-source/SKILL.mddocumenting the canonicalmodify_source.shpattern. The lookaside URL embeds the SHA-512 of the served file, so the repack must be byte-deterministic. Covers:tarflags (--sort=name,--owner=0,--group=0,--numeric-owner,--mtime='@<epoch>',--format=gnu,LC_ALL=C).xzflags (-T 1,-9).xzwithout--block-size,mktempfor output filename,gzipwithout-n,find -newer, etc.).3.
skill-signing-failure-remediationskillNew skill at
.github/skills/skill-signing-failure-remediation/SKILL.mddocumenting the decision tree:spec-remove-subpackage(orbuild.withoutif a clean bcond exists).replace-upstreampattern with amodify_source.shscript (seeskill-modify-source).Always run a reverse-dependency scan before choosing (a) or (b).
4. Render/lock cycle ordering (
copilot-instructions.md,skill-update-component)Canonical sequence: edit
comp.toml→azldev comp update -p <name>→ commit (toml + lock) →azldev comp render -p <name>→ amend (rendered spec). Keeps the lock fingerprint,%changelog, andRelease:in sync within a single commit. The existingskill-update-componenthad this pattern; this PR clarifies the commit order explicitly.5. GPG signing (
copilot-instructions.md)Every commit MUST be GPG-signed. Verify with
git log -1 --show-signature.6. Public-content hygiene (
copilot-instructions.md,comp-toml.instructions.md)Overlay descriptions,
replace-reasonstrings, comp.toml comments, spec files,modify_source.shecho lines, commit messages, and PR descriptions describe motivations technically and neutrally — no specific internal infrastructure names (signing services, scanner brand names, internal pipeline/tooling names, internal Azure tenants, etc.). Use neutral phrasing.Files
.github/copilot-instructions.md.github/instructions/comp-toml.instructions.mdreplace-upstreamsource-override pattern, public-content hygiene,components.tomlfile-organization rules..github/skills/skill-modify-source/SKILL.md.github/skills/skill-signing-failure-remediation/SKILL.md.github/skills/skill-update-component/SKILL.mdAGENTS.mdbase/comps/AGENTS.mdCommits
instructions(comp-toml): document replace-upstream source override patternskills: add skill-modify-source for byte-deterministic Source0 repackinstructions: clarify render order, GPG signing, public-content hygiene, and signing-failure remediation skillAll commits are GPG-signed.