Report render comparison artifacts from CI#876
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds CI support for collecting and surfacing render snapshot comparison artifacts when render verification fails, including downloadable reports and PR-visible links.
Changes:
- Adds PowerShell tooling to collect
.received.png/.verified.png/.diff.png(+ metadata) into a structured artifact withindex.html,README.md, andmanifest.json. - Updates CI workflow to upload the artifact, add a summary link, and maintain a single PR comment with run/artifact links.
- Adds a local smoke-test script for the artifact collector.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| Src/Common/RootSite/RootSiteTests/RenderBenchmarkTestsBase.cs | Modifies sample render text (currently causes intentional render diffs). |
| Build/Agent/Test-CollectRenderArtifacts.ps1 | Adds a local smoke test harness for the artifact collection script. |
| Build/Agent/Collect-RenderArtifacts.ps1 | New artifact collector + report/manifest generation and GitHub output wiring. |
| .github/workflows/CI.yml | Uploads the artifact on failure and posts/updates a PR comment + workflow summary. |
Render comparison artifactsNo render snapshot failures were captured in the latest successful CI run. Latest run: b1fdb75c831e run 25688716247.1. |
jasonleenaylor
left a comment
There was a problem hiding this comment.
Did you investigate if reg-actions can handle the reporting when we have detected the diffs?
@jasonleenaylor reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: 1 of 5 files reviewed, all discussions resolved.
|
Decision note for this PR: I kept the current render-artifact collection flow and only swapped the sticky PR comment maintenance to What I considered:
Why this change:
So the current direction is:
|
Render comparison artifactsNo render snapshot failures were captured in the latest successful CI run. Latest run: c38dcfe86533 run 25742517199.1. |
Adds CI support for render snapshot failure artifacts.
.received.png,.verified.png,.diff.png, and metadata intoOutput/RenderArtifactsindex.html,README.md, andmanifest.jsonLocal verification:
./Build/Agent/Test-CollectRenderArtifacts.ps1This change is