Skip to content

(CAT-2632) Enable safe fork-PR CI against private puppetcore#163

Merged
gavindidrichsen merged 8 commits into
mainfrom
CAT-2632-enable_safe_fork_pr_ci
Jun 8, 2026
Merged

(CAT-2632) Enable safe fork-PR CI against private puppetcore#163
gavindidrichsen merged 8 commits into
mainfrom
CAT-2632-enable_safe_fork_pr_ci

Conversation

@LukasAud

@LukasAud LukasAud commented May 12, 2026

Copy link
Copy Markdown
Contributor

Adds a two-track CI pattern so fork PRs can run acceptance tests against the private Puppet Forge under layered controls.

  • New reusable fork_ci_label_guard.yml strips the allowed-for-ci label on every fork-PR synchronize, so each new commit re-requires a fresh CODEOWNER review and re-label before the privileged acceptance workflow can run.
  • module_acceptance.yml: acceptance job conditionally targets the puppetcore-fork GitHub Environment (per-run reviewer approval) for fork PRs only; the second checkout now uses the PR head SHA with persist-credentials disabled; env-var passthrough applied to all user-controlled interpolations in run blocks; third-party actions pinned to SHAs.
  • module_ci.yml: matching hardening (permissions, SHA pins, persist-credentials, env passthrough). Both reusable workflows gain a defense-in-depth if: so unlabeled fork PRs cannot run under pull_request_target.
  • example/module/ci.yml: rewritten to the two-track pattern (pull_request for Spec without secrets; pull_request_target gated by label for Acceptance). synchronize intentionally omitted from pull_request_target types to avoid racing with the label guard.
  • New docs/how-to/fork-pr-ci.md covers per-module adoption (label, ruleset, environment, branch protection) and the reviewer checklist for files that warrant scrutiny before applying the label.

Checklist

  • 🟢 Spec tests.
  • 🟢 Acceptance tests.
  • Manually verified.

Adds a two-track CI pattern so fork PRs can run acceptance tests against
the private Puppet Forge under layered controls.

- New reusable fork_ci_label_guard.yml strips the `allowed-for-ci`
  label on every fork-PR synchronize, so each new commit re-requires a
  fresh CODEOWNER review and re-label before the privileged acceptance
  workflow can run.
- module_acceptance.yml: acceptance job conditionally targets the
  `puppetcore-fork` GitHub Environment (per-run reviewer approval) for
  fork PRs only; the second checkout now uses the PR head SHA with
  persist-credentials disabled; env-var passthrough applied to all
  user-controlled interpolations in run blocks; third-party actions
  pinned to SHAs.
- module_ci.yml: matching hardening (permissions, SHA pins,
  persist-credentials, env passthrough). Both reusable workflows gain
  a defense-in-depth `if:` so unlabeled fork PRs cannot run under
  pull_request_target.
- example/module/ci.yml: rewritten to the two-track pattern
  (pull_request for Spec without secrets; pull_request_target gated by
  label for Acceptance). `synchronize` intentionally omitted from
  pull_request_target types to avoid racing with the label guard.
- New docs/how-to/fork-pr-ci.md covers per-module adoption (label,
  ruleset, environment, branch protection) and the reviewer checklist
  for files that warrant scrutiny before applying the label.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@LukasAud LukasAud requested review from a team as code owners May 12, 2026 08:02
LukasAud and others added 6 commits May 12, 2026 09:13
GitHub rulesets target branches/tags/pushes, not labels. Label
add/remove is gated by repository role (Triage+ can label; external
Read cannot). Replace the doc's ruleset step with a repo-role audit
step, and flag a `labeller_verification.yml` reusable workflow as the
recommended follow-up for strict per-label enforcement.

Also clarify environment-secret handling: GitHub resolves secrets with
environment > repo > org precedence, so org-level secrets fall through
to the env automatically and only need duplication if they currently
live at the repo level. Update troubleshooting accordingly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit bundled stylistic hardening (SHA-pinning third-party
actions, explicit `repository:` on checkouts, env-passthrough for trusted
`inputs.*` values, refactored Install Puppet agent if/else) alongside the
security work. Larger diff, harder to troubleshoot in the long run.

Reverts those stylistic changes. Keeps only the controls that close a
concrete fork-PR attack path:

- `permissions: contents: read` on both reusable workflows.
- Defense-in-depth `if:` on `setup_matrix` so unlabelled fork PRs cannot
  run under `pull_request_target`.
- Conditional `environment: puppetcore-fork` on the acceptance job
  (fork PRs only).
- Explicit `ref: github.event.pull_request.head.sha` and
  `persist-credentials: false` on every checkout. `ref` is required
  because `pull_request_target` defaults to the base ref;
  `persist-credentials: false` stops PR-controlled scripts from reading
  GITHUB_TOKEN out of .git/config.
- Env-passthrough only for `matrix.platforms.provider`,
  `matrix.platforms.image`, and `matrix.collection.*` — these flow from
  PR-controlled metadata.json via matrix_from_metadata_v3 and are real
  shell-injection vectors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without this, a fork contributor could: open a PR, get it labelled and
approved against SHA A, close the PR (label persists — close doesn't
fire synchronize), push SHA B (no events fire on a closed PR), then
reopen. The reopened event payload still carries the label, so
Acceptance.if evaluates true against the unreviewed SHA B.

Adds `closed` to fork_ci_label_guard.yml's trigger types so the label
is stripped on close. After close, any later reopen sees an empty
labels array in its event payload and skips Acceptance; running it
again requires a CODEOWNER to re-review and re-label.

Stripping on `reopened` doesn't work: event-payload labels are
snapshotted at trigger time, so the main caller's Acceptance.if sees
the label as present in parallel with the guard removing it.

Side effect: closing a merged PR strips the label too. Cosmetic — the
label is meaningful only on open PRs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move `environment: puppetcore-fork` off the matrix `acceptance` job
and onto a new non-matrix `approval_gate` job that the matrix depends
on. Each matrix entry previously declared the environment, so a
labelled fork PR with N platforms produced 2N "temporarily deployed"
timeline events; with the gate centralised, fork PRs now produce
exactly 2 deployment events per run regardless of matrix size.

Behaviour:
- Fork PR via pull_request_target: approval_gate runs, awaits env
  approval, then matrix acceptance proceeds.
- Same-repo PR / workflow_dispatch / pull_request: approval_gate's
  if-expression is false → job is skipped; acceptance handles the
  skipped case via `always() && (success || skipped)` in its if.
- Non-updated callers (existing pull_request-only): same — gate
  skipped, acceptance runs.

The approval prompt now appears against a clearly-named single job
rather than the first matrix entry; the gate's only step echoes the
PR number and head SHA so the reviewer can verify what they're
approving (mitigates the stale-queued-run SHA-drift scenario).

Caller workflows do not need changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
matrix.collection can be either a plain string or a mapping. Binding the
mapping form directly to an env value fails template validation with
"A mapping was not expected". Mirror the existing matrix.collection.collection
|| matrix.collection fallback so the env value is always a scalar string.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-applies SHA pinning for the third-party actions touched by this PR.
A tag pin (`@v4`, `@v1`) is mutable — anyone who can push to the action
repo can move the tag to malicious code, and consumers ship that on
the next run. SHA pins are immutable and close the supply-chain vector.

Pins (current latest release at time of writing):

- actions/checkout                @df4cb1c069e1874edd31b4311f1884172cec0e10  # v6.0.3
- ruby/setup-ruby                 @afeafc3d1ab54a631816aba4c914a0081c12ff2f  # v1.310.0
- twingate/github-action          @f1e69fc5af67b737d8597b759e5c607f567e77f2  # v1.8
- reviewdog/action-shellcheck     @4c07458293ac342d477251099501a718ae5ef86e  # v1.32.0

actions/checkout jumps from v4 to v6.0.3. Our usage (`ref:` +
`persist-credentials: false`) is unaffected by the changes between
those majors, but worth knowing if anything breaks during pilot.

Trailing version comments preserve readability. Dependabot's
`github-actions` ecosystem config already covers this repo and will
offer SHA bumps for new releases on its weekly cycle.

Scope: only the workflows touched by this PR. Other workflows in this
repo (mend_ruby.yml, gem_*.yml, module_release*.yml, etc.) still use
tag pins and should be SHA-pinned as a separate hardening pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@LukasAud LukasAud added the feature New feature or request label Jun 3, 2026
Per reviewer feedback. `gh pr edit ... || true` masked not just the
intended 404 "label not present" case but also auth / rate-limit / API
failures — any of which would mark the workflow green while the strip
never actually happened. The fork-PR security model relies on this
strip executing, so silent failure is the mode we shouldn't mask.

Check then act: same end behaviour for absent-label (skip with a log
line), but genuine failures now surface as red workflow runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@gavindidrichsen gavindidrichsen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — clean two-track design, and the threat model in the how-to is thorough. Left two optional mermaid diagrams inline (two-track flow + label lifecycle)
to make fork-pr-ci.md legible at a glance — take or leave them.

Comment thread .github/workflows/fork_ci_label_guard.yml Outdated
Comment thread docs/how-to/fork-pr-ci.md

The Spec track has already run by the time the reviewer sees the PR; review its result alongside the diff.

## How it works

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A diagram here would make the two-track model legible at a glance — it's the reviewer's mental model for the whole design. Suggesting it right under the heading so the prose bullets below still read as the step-by-step detail:

Suggested change
## How it works
## How it works
```mermaid
flowchart TD
PR([Fork PR opened / updated]) --> spec
subgraph public["Spec track — pull_request (no secrets)"]
spec[Spec: rake spec]
end
subgraph privileged["Acceptance track — pull_request_target (secrets in scope)"]
guard{"allowed-for-ci<br/>label present?"}
review[["CODEOWNER reviews diff:<br/>Rakefile, spec/**, Gemfile,<br/>.fixtures.yml, metadata.json"]]
env["approval_gate job<br/>→ puppetcore-fork environment<br/>(reviewer compares head SHA)"]
accept[Acceptance: litmus:*<br/>PUPPET_FORGE_TOKEN in scope]
end
PR --> guard
guard -- no --> stop1([No acceptance run])
review -- applies label --> guard
guard -- yes / labeled --> env
env -- approved --> accept
```

Comment thread docs/how-to/fork-pr-ci.md
- A fork contributor opens a PR. `Spec` runs immediately (public track, no secrets). `Acceptance` does not run.
- A CODEOWNER reviews the diff, focusing on the files above, and applies `allowed-for-ci`. The `labeled` event triggers the workflow on `pull_request_target`. A single `approval_gate` job (non-matrix) targets the `puppetcore-fork` environment and pauses for environment approval — the approval prompt shows the head SHA being deployed, which the reviewer should compare against the SHA they reviewed before approving. A `@puppetlabs/modules` member approves; `approval_gate` completes; the matrix `acceptance` jobs then run with `PUPPET_FORGE_TOKEN` in scope.
- If the contributor pushes a new commit, `fork_ci_label_guard.yml` fires on `synchronize` and removes the label. The main caller workflow does **not** include `synchronize` in its `pull_request_target` trigger, so no acceptance run is created for the new commit. To run acceptance against the new code, a CODEOWNER must review again and re-apply the label.
- If the PR is closed, `fork_ci_label_guard.yml` also fires on `closed` and removes the label. This prevents an attack where a contributor closes the PR, pushes new commits (no events fire on a closed PR), then reopens — without close-strip the label would persist across the reopen and the new commits would inherit the prior authorisation. Side effect: closing a merged PR also strips the label, which is cosmetic (the label is only meaningful for open PRs).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bullet describes the subtlest control in the design — the close→push→reopen replay and why the closed trigger defeats it. A state diagram of the label lifecycle makes it reviewable at a glance, and it maps 1:1 to fork_ci_label_guard.yml (the file this PR adds). Suggesting it right after this bullet:

Suggested change
- If the PR is closed, `fork_ci_label_guard.yml` also fires on `closed` and removes the label. This prevents an attack where a contributor closes the PR, pushes new commits (no events fire on a closed PR), then reopens — without close-strip the label would persist across the reopen and the new commits would inherit the prior authorisation. Side effect: closing a merged PR also strips the label, which is cosmetic (the label is only meaningful for open PRs).
- If the PR is closed, `fork_ci_label_guard.yml` also fires on `closed` and removes the label. This prevents an attack where a contributor closes the PR, pushes new commits (no events fire on a closed PR), then reopens — without close-strip the label would persist across the reopen and the new commits would inherit the prior authorisation. Side effect: closing a merged PR also strips the label, which is cosmetic (the label is only meaningful for open PRs).
```mermaid
stateDiagram-v2
[*] --> Unlabeled
Unlabeled --> Labeled: CODEOWNER applies<br/>allowed-for-ci (after review)
Labeled --> Acceptance: pull_request_target<br/>type=labeled → approval gate
Labeled --> Unlabeled: synchronize (new commit)<br/><b>guard strips label</b>
Labeled --> Unlabeled: closed<br/><b>guard strips label</b>
note right of Unlabeled
closed-strip defeats the reopen attack:
close → push (no events fire) → reopen
would otherwise carry the stale label
onto unreviewed commits.
end note
Acceptance --> Unlabeled: next synchronize<br/>re-requires fresh review
```

@gavindidrichsen gavindidrichsen merged commit 0237b02 into main Jun 8, 2026
3 checks passed
@gavindidrichsen gavindidrichsen deleted the CAT-2632-enable_safe_fork_pr_ci branch June 8, 2026 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants