Skip to content

test(snapshot): Add structured JSON fixture targets#338

Merged
cameroncooke merged 9 commits intomainfrom
cameroncooke/test/structured-json-fixtures
Apr 14, 2026
Merged

test(snapshot): Add structured JSON fixture targets#338
cameroncooke merged 9 commits intomainfrom
cameroncooke/test/structured-json-fixtures

Conversation

@cameroncooke
Copy link
Copy Markdown
Collaborator

Add a structured JSON fixture corpus for the planned CLI --output json mode.

This introduces a target fixture tree under src/snapshot-tests/__fixtures__/json and a companion planning doc in docs/dev/STRUCTURED_JSON_OUTPUT_PLAN.md. The fixtures are normalized around shared schema families so we can review the output contract before wiring the producer and snapshot harness to it.

The main goal here is to lock down the result shapes early: text and current event-stream output stay unchanged for now, while the future structured JSON mode gets a concrete, reviewable contract across simulator, device, macOS, SwiftPM, debugging, discovery, session-management, and UI automation workflows.

I kept this PR fixture-only. It does not update the runtime producer, tests, or harness plumbing yet.

Add a structured JSON fixture corpus for the planned CLI json output mode.

Define shared envelope and schema-family targets in docs/dev/STRUCTURED_JSON_OUTPUT_PLAN.md and capture the expected payloads across simulator, device, macOS, SwiftPM, debugging, discovery, and session workflows.

This keeps the current producer unchanged while giving the upcoming implementation a concrete fixture contract to build against.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Apr 13, 2026

Open in StackBlitz

npm i https://pkg.pr.new/xcodebuildmcp@338

commit: 9fbdebf

@cameroncooke cameroncooke marked this pull request as ready for review April 13, 2026 17:50
Copy link
Copy Markdown
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Device-list schema missing isAvailable field in resources fixture
    • Added the missing isAvailable field to all device entries in resources/devices--success.json to match the device-list schema contract.
  • ✅ Fixed: Simulator success fixtures inconsistently omit diagnostics field
    • Added the missing diagnostics field with empty warnings and errors arrays to simulator success fixtures for stop, install, and launch-app operations.

Create PR

Or push these changes by commenting:

@cursor push d6600841ee
Preview (d6600841ee)
diff --git a/src/snapshot-tests/__fixtures__/json/resources/devices--success.json b/src/snapshot-tests/__fixtures__/json/resources/devices--success.json
--- a/src/snapshot-tests/__fixtures__/json/resources/devices--success.json
+++ b/src/snapshot-tests/__fixtures__/json/resources/devices--success.json
@@ -6,32 +6,36 @@
   "data": {
     "devices": [
       {
-        "name": "Cameron’s iPhone 16 Pro Max",
+        "name": "Cameron's iPhone 16 Pro Max",
         "deviceId": "<UUID>",
         "platform": "iOS",
         "osVersion": "26.3.1 (a)",
-        "state": "<STATUS>"
+        "state": "<STATUS>",
+        "isAvailable": true
       },
       {
         "name": "iPhone",
         "deviceId": "<UUID>",
         "platform": "iOS",
         "osVersion": "26.1",
-        "state": "<STATUS>"
+        "state": "<STATUS>",
+        "isAvailable": false
       },
       {
-        "name": "Cameron’s Apple Watch",
+        "name": "Cameron's Apple Watch",
         "deviceId": "<UUID>",
         "platform": "watchOS",
         "osVersion": "10.6.1",
-        "state": "<STATUS>"
+        "state": "<STATUS>",
+        "isAvailable": false
       },
       {
-        "name": "Cameron’s Apple Watch",
+        "name": "Cameron's Apple Watch",
         "deviceId": "<UUID>",
         "platform": "watchOS",
         "osVersion": "26.3",
-        "state": "<STATUS>"
+        "state": "<STATUS>",
+        "isAvailable": true
       }
     ]
   }

diff --git a/src/snapshot-tests/__fixtures__/json/simulator/install--success.json b/src/snapshot-tests/__fixtures__/json/simulator/install--success.json
--- a/src/snapshot-tests/__fixtures__/json/simulator/install--success.json
+++ b/src/snapshot-tests/__fixtures__/json/simulator/install--success.json
@@ -10,6 +10,10 @@
     "artifacts": {
       "simulatorId": "<UUID>",
       "appPath": "<HOME>/Library/Developer/XcodeBuildMCP/DerivedData/Build/Products/Debug-iphonesimulator/CalculatorApp.app"
+    },
+    "diagnostics": {
+      "warnings": [],
+      "errors": []
     }
   }
 }

diff --git a/src/snapshot-tests/__fixtures__/json/simulator/launch-app--success.json b/src/snapshot-tests/__fixtures__/json/simulator/launch-app--success.json
--- a/src/snapshot-tests/__fixtures__/json/simulator/launch-app--success.json
+++ b/src/snapshot-tests/__fixtures__/json/simulator/launch-app--success.json
@@ -13,6 +13,10 @@
       "processId": "<PID>",
       "runtimeLogPath": "<HOME>/Library/Developer/XcodeBuildMCP/logs/io.sentry.calculatorapp_<TIMESTAMP>_pid<PID>.log",
       "osLogPath": "<HOME>/Library/Developer/XcodeBuildMCP/logs/io.sentry.calculatorapp_oslog_<TIMESTAMP>_pid<PID>.log"
+    },
+    "diagnostics": {
+      "warnings": [],
+      "errors": []
     }
   }
 }

diff --git a/src/snapshot-tests/__fixtures__/json/simulator/stop--success.json b/src/snapshot-tests/__fixtures__/json/simulator/stop--success.json
--- a/src/snapshot-tests/__fixtures__/json/simulator/stop--success.json
+++ b/src/snapshot-tests/__fixtures__/json/simulator/stop--success.json
@@ -10,6 +10,10 @@
     "artifacts": {
       "simulatorId": "<UUID>",
       "bundleId": "io.sentry.calculatorapp"
+    },
+    "diagnostics": {
+      "warnings": [],
+      "errors": []
     }
   }
 }

This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

@cameroncooke
Copy link
Copy Markdown
Collaborator Author

@cursor push d660084

- Add missing isAvailable field to device-list schema in resources/devices--success.json
- Add missing diagnostics field to simulator success fixtures (stop, install, launch-app)

Applied via @cursor push command
…form schema contract

- Add missing summary.durationMs and summary.target to all simulator fixtures
- Fix singular "Test failed" to "Tests failed" in simulator failure fixtures
- Normalize counts key ordering to passed/failed/skipped in simulator test failure
- Align resources/devices field ordering with device/list fixture
cameroncooke and others added 4 commits April 13, 2026 20:36
…cross all fixtures

- durationMs: "<DURATION>" -> 1234
- processId/pid: "<PID>" -> 99999
- Test failure counts: "<PASS_COUNT>"/"<FAIL_COUNT>"/"<SKIP_COUNT>" -> mock integers matching discovered totals
- session-status: uptimeMs/rssBytes/heapUsedBytes -> mock integers
- doctor: manifest counts, simulatorVideoCaptureSupported -> proper types
- stack: threadId/index -> integers
- swift-package/list: rename uptime -> uptimeSeconds
- Fix remaining "Test failed" -> "Tests failed" in macos and swift-package error fixtures
…drop resource fixtures

- Convert debug variables scopes from array to keyed object, remove displayName
- Remove output.stdout/stderr from build-and-run success fixtures
- Remove remainingCount from test failure discovered items
- Delete resource fixtures (devices, doctor, session-status, simulators)
Add a dedicated schema corpus for structured JSON fixtures and validate
each discovered fixture against its declared schema and version.

This gives the hand-crafted fixture contracts an executable check
without coupling them to the existing text snapshot harness.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
Document schema versioning and deployment policy and add a
workflow that syncs published schema files into the website
repository for Vercel deployment.

This keeps schema authoring in this repository while making the
published schema URLs durable and real.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
Copy link
Copy Markdown
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 102658c. Configure here.

Limit the publish workflow token to read-only contents access and
avoid interpolating the deploy key secret directly inside the shell
guard.

This keeps the workflow aligned with GitHub Actions security
expectations and removes a brittle secret-handling pattern.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
@cameroncooke cameroncooke merged commit c40789b into main Apr 14, 2026
12 checks passed
@cameroncooke cameroncooke deleted the cameroncooke/test/structured-json-fixtures branch April 14, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants