Collect coverage only on the primary CI job; use the sys.monitoring coverage core#264
Conversation
All 10 matrix jobs ran pytest with --cov, but only the primary job (ubuntu 3.12) uploads the reports and feeds the PR comment — the other 9 computed coverage and discarded it, at ~65% extra pytest wall time (worst on Windows, where the unit step took 734s vs 161s on ubuntu). Coverage flags move to job-level COV/COV_APPEND env vars that resolve to empty on non-primary jobs, preserving the --cov-append chain across the unit/TUI/browser/benchmark steps on the primary job. Measured impact on reported coverage: none. Diffing covered lines (local Windows full run vs the primary artifact of run 28703179425, same commit) shows Windows uniquely covers only 5 lines — the SIGUSR1 no-op return and the drive-letter branch of convert_project_path_to_claude_dir in cli.py — and those were already misses in the published report, which has always been ubuntu-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ivalent) Set core = "sysmon" in [tool.coverage.run] rather than a CI env var: one source of truth that also speeds up local `just test-cov` and de-flakes test_render_performance_under_threshold, which can exceed its threshold under the C tracer's overhead. Measured on the CI unit-test selection: +67% wall time under the C tracer vs +14% under sysmon. On Py < 3.12 coverage emits a single cosmetic no-sysmon warning and falls back to the default core — moot in CI, where the only coverage job (primary, ubuntu 3.12) is on 3.12+. Bump the coverage floor to 7.9, which introduced the [run] core setting; older versions warn 'unrecognized option' and silently fall back to the slow default core, losing the speedup. uv.lock already resolved 7.12.0, so only non-uv installs are affected by the floor. COVERAGE_CORE env var still overrides the setting for A/B timing (e.g. COVERAGE_CORE=ctrace). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe CI workflow now derives pytest coverage flags from job-level environment variables (COV, COV_APPEND) enabled only on the primary matrix job, replacing hardcoded flags. pyproject.toml sets Coverage.py's sysmon core and raises the coverage dependency minimum version to 7.9. ChangesCI Coverage Configuration
Estimated code review effort: 1 (Trivial) | ~5 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/ci.yml (1)
53-69: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low valueSilence zizmor template-injection warnings by using shell env vars instead of
${{ env.* }}.Since
COV/COV_APPENDare already exported as job-levelenv:vars, therun:steps can reference them as shell variables ($COV,$COV_APPEND) instead of GitHub Actions expression syntax. This is the documented zizmor remediation fortemplate-injection: expression expansion happens before the shell sees the script, whereas shell variable substitution goes through normal quoting rules. Although the actual risk here is low (values are derived only frommatrix.is-primary, not attacker input), this removes all 4 flagged findings with a trivial change.🔧 Proposed fix
- name: Run unit tests (coverage on primary only) # -p no:playwright: unit tests don't use the browser fixtures; skipping the # plugin avoids a ~1s playwright import per xdist worker, which Windows pays # per spawned worker. See work/xdist-import-cost.md. - run: uv run pytest -p no:playwright -m "not (tui or browser or benchmark)" ${{ env.COV }} + run: uv run pytest -p no:playwright -m "not (tui or browser or benchmark)" $COV - name: Run TUI tests (coverage append on primary only) - run: uv run pytest -m tui ${{ env.COV_APPEND }} + run: uv run pytest -m tui $COV_APPEND - name: Run browser tests (coverage append on primary only) - run: uv run pytest -m browser ${{ env.COV_APPEND }} + run: uv run pytest -m browser $COV_APPEND - name: Run benchmark tests with coverage append (primary only) if: matrix.is-primary env: CLAUDE_CODE_LOG_DEBUG_TIMING: "1" - run: uv run pytest -m benchmark ${{ env.COV_APPEND }} -v + run: uv run pytest -m benchmark $COV_APPEND -v🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 53 - 69, The pytest workflow steps still interpolate COV and COV_APPEND with GitHub Actions expressions, which triggers zizmor template-injection warnings; update the Run unit tests, Run TUI tests, Run browser tests, and Run benchmark tests steps to reference the already-exported shell environment variables instead. In the ci workflow, keep the job-level env definitions for COV/COV_APPEND and change the run commands to use shell expansion in those step scripts so the values are resolved by the shell rather than by `${{ env.* }}`.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 53-69: The pytest workflow steps still interpolate COV and
COV_APPEND with GitHub Actions expressions, which triggers zizmor
template-injection warnings; update the Run unit tests, Run TUI tests, Run
browser tests, and Run benchmark tests steps to reference the already-exported
shell environment variables instead. In the ci workflow, keep the job-level env
definitions for COV/COV_APPEND and change the run commands to use shell
expansion in those step scripts so the values are resolved by the shell rather
than by `${{ env.* }}`.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 920941a7-7574-4ca1-8812-d3f86d745b17
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (2)
.github/workflows/ci.ymlpyproject.toml
|
Windows 3.12 and its 21m... Fable called that a "lemon VM" and losing at the Windows runner lottery :P But hey, the Linux numbers did drop again, so I guess we're good with these changes (initially aimed at reducing the gap between Windows and Linux build times!) |
|
(Claude) Re the zizmor
Making shell expansion safe would need |
Part of #248.
Problem
All 10 CI matrix jobs ran pytest with
--cov(xml + html + term, across the unit/TUI/browser step chain), but only the primary job (ubuntu 3.12) uploads the reports and feeds the PR coverage comment — the other 9 computed coverage and discarded it. Measured cost of those flags on the CI unit-test selection: +67% pytest wall time under the default C tracer (worst on Windows, where the unit step ran 734s vs 161s on ubuntu, run 28703179425).Changes
COV/COV_APPENDenv vars that resolve to empty on non-primary jobs, preserving the--cov-appendchain (unit → TUI → browser → benchmark) on the primary job. Workflow validated with actionlint.core = "sysmon"in[tool.coverage.run]— the sys.monitoring core (Py 3.12+) cuts the remaining coverage tax on the primary job from +67% to +14%, with byte-identical coverage totals (84.71%). On Py < 3.12 coverage warns once and falls back to the default core; moot in CI since the only coverage job is on 3.12. Side benefit: de-flakestest_render_performance_under_threshold, which can exceed its threshold under the C tracer.COVERAGE_CORE=ctracestill overrides for A/B timing.[run] coresetting; older versions warn and silently keep the slow core). uv.lock already resolves 7.12.0.Does dropping Windows coverage lose anything?
No. Diffing covered lines (full local Windows run vs the primary job's coverage artifact at the same commit) shows Windows uniquely covers exactly 5 lines, all in
cli.py: the SIGUSR1 no-op return and the drive-letter branch ofconvert_project_path_to_claude_dir. Those were already misses in the published report, which has always been ubuntu-only — so reported coverage changes by exactly 0 lines. (If cross-OS merged coverage is ever wanted, the groundwork is noted in the ci.yml comment;relative_files = trueis already set.)Expected effect
Verification on this PR's CI run
--covflags in the log🤖 Generated with Claude Code
Summary by CodeRabbit