fix: recover structured output for /codex:adversarial-review when the turn uses tools

[qinjie]

Problem

/codex:adversarial-review fails with Unexpected token … in JSON at position 0 on every non-trivial review. When Codex investigates the diff before answering, codex-cli 0.142.3 ends the outputSchema turn without emitting the forced final_answer, so the companion captures a phase: "commentary" preamble and parseStructuredOutput's bare JSON.parse throws.

Plain /codex:review is unaffected — it uses the native reviewer (exitedReviewMode → reviewText), no outputSchema, no JSON.parse.

What the user sees

# Codex Adversarial Review

Codex did not return valid structured JSON.

- Parse error: Unexpected token I in JSON at position 0

Raw final message:

I'll inspect the HEAD~2..HEAD diff directly and trace the changed runtime paths …

Turn duration is suspiciously short (~9–12s): Codex investigates, the turn completes, but no structured final_answer is ever emitted.

Root cause

Driving codex app-server directly (initialize → thread/start → turn/start with an outputSchema that has a verdict property) shows:

• No tool use ("reply with approval") → item/completed { agentMessage, phase:"final_answer", text:'{"verdict":"approve",…}' } ✅ schema works.
• With tool use ("read file X, then return JSON") → agentMessage phase=commentary "I'll read…" → several commandExecution items → turn/completed with no final_answer ❌.

So this is a codex-cli behavior (forced-schema output dropped when one turn mixes tool calls + outputSchema), but the plugin makes it fatal because recordItem() sets lastAgentMessage for every agentMessage (including non-final commentary), and parseStructuredOutput hard-fails on prose.

Fix

In plugins/codex/scripts/lib/codex.mjs:

1. Two-turn elicitation in runAppServerTurn — new opt-in structuredRetryPrompt option. If a schema was requested but the captured message isn't JSON once the turn completes, resume the same thread with a JSON-only prompt (retry budget structuredRetryAttempts, default 3). Same-thread resume retains the investigation context, so the model emits the verdict it skipped. Guarded by !looksLikeJson(finalMessage), so it never fires once the CLI emits final_answer directly — a forward-compatible no-op once the upstream CLI bug is fixed.
2. parseStructuredOutput tolerates a fenced ```json block via a small stripJsonFences helper (also used by `looksLikeJson`).

In plugins/codex/scripts/codex-companion.mjs:

3. The adversarial-review call site passes structuredRetryPrompt. The task path and /codex:review are untouched.

Maintainer note: a cleaner long-term variant of (1) is to prefer the last agentMessage with phase === "final_answer" when populating lastAgentMessage/finalMessage, so non-final commentary can never clobber the answer. That's included in spirit but is necessary-but-not-sufficient on 0.142.3 — no final_answer is emitted at all after tool use, so the resume turn is what actually produces the JSON. Happy to reshape toward whichever approach you prefer.

Tests

• tests/fake-codex-fixture.mjs: new adversarial-tooluse-then-json behavior that, on the first outputSchema turn, emits a commentary preamble + a commandExecution and completes with no final_answer; the follow-up JSON-only turn emits the structured final_answer. This reproduces the 0.142.3 protocol exactly.
• tests/runtime.test.mjs: asserts adversarial-review exits 0 and renders Verdict: (not "did not return valid structured JSON") under that behavior.

Verified fail-before / pass-after: the new test fails on main with the exact parse error and passes with this change. The rest of the suite's pass/fail set is unchanged by this PR.

npm test

Risk / scope

• Opt-in via structuredRetryPrompt; the task path and /codex:review are untouched.
• Forward-compatible: becomes a no-op if codex-cli later emits final_answer after tool use.
• Adds one extra LLM turn (up to N retries) only on the adversarial path and only when the first turn produced no JSON.

---

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d87a86dcc9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Preserve failures from the investigation turn

When the first schema turn ends as failed or interrupted after producing only a commentary message, this retry path still runs and then replaces the original failed finalTurn with the retry turn. If that JSON-only retry completes, the later status calculation reports success, so /codex:adversarial-review can render a normal verdict and exit 0 even though the investigation turn actually failed. Please only retry after a completed first turn, or keep the overall status non-zero whenever turnState.finalTurn.status was not completed.

Useful? React with 👍 / 👎.