Fix Codex config generation for current @openai/codex#7
Conversation
|
Warning Review limit reached
Next review available in: 47 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe TEMPLATE constant in codex.py was modified to embed model and model_provider settings directly instead of using a profile-based configuration, and the associated profile section was removed. Additionally, codex_create_toml now strips trailing slashes from server_url before formatting. ChangesCodex TOML Generation Update
Estimated code review effort: 1 (Trivial) | ~3 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/coding_agent_bench/helpers/codex.py (1)
22-22: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winUnescaped interpolation into TOML string.
model_nameandserver_urlare interpolated into a TOML-formatted string via.format()without escaping quotes/special characters. Per the referencedCreateJobRequestinsrc/coding_agent_bench/api.py, both values originate from an external API request body, so a value containing a"would produce invalid/broken TOML (or in principle alter adjacent keys).🛡️ Suggested fix using a TOML-safe encoder
+import tomlkit + def codex_create_toml(model_name: str, server_url: str, outpath: Path): - toml = TEMPLATE.format(model_name=model_name, server_url=server_url.rstrip("/")) + toml = TEMPLATE.format( + model_name=tomlkit.string(model_name).as_string(), + server_url=server_url.rstrip("/"), + )Also flagging the static analysis hint on Line 23 (
open(outpath, "w")) as a likely false positive here, sinceoutpathis constructed internally (e.g.Path("config.toml").absolute()inconfigs.py) rather than taken directly from request input.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/coding_agent_bench/helpers/codex.py` at line 22, The TOML template interpolation in codex.py is unsafe because model_name and server_url come from CreateJobRequest and are inserted into TEMPLATE via .format() without TOML escaping. Update the code path that builds toml so these values are serialized with a TOML-safe encoder or otherwise properly escaped before formatting, and keep the open(outpath, "w") usage unchanged since outpath is internally constructed and not user-controlled.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/coding_agent_bench/helpers/codex.py`:
- Line 22: The TOML template interpolation in codex.py is unsafe because
model_name and server_url come from CreateJobRequest and are inserted into
TEMPLATE via .format() without TOML escaping. Update the code path that builds
toml so these values are serialized with a TOML-safe encoder or otherwise
properly escaped before formatting, and keep the open(outpath, "w") usage
unchanged since outpath is internally constructed and not user-controlled.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Enterprise
Run ID: b316313a-fe19-4336-8a03-e82365d0e9d5
📒 Files selected for processing (1)
src/coding_agent_bench/helpers/codex.py
1 participant
Summary
This PR updates the Codex helper config generation to avoid the legacy Codex profile format that is rejected by current
@openai/codexreleases.Previously, the generated
config.tomlused the legacy top-level profile selector:and a legacy profile section:
Current Codex CLI rejects this format and exits before producing a Codex session.
Problem
When running
coding-agent-benchwith--agent codexand a local OpenAI-compatible server, the benchmark installs Codex with:With the current Codex CLI, the generated config fails with:
The benchmark then reports:
Change
This PR updates
src/coding_agent_bench/helpers/codex.pyso the generated config uses the current direct provider format instead of the legacy profile format.The generated config now uses:
instead of:
Tested
Tested locally on macOS with Docker Desktop, Ollama, and the Codex agent.
Models tested:
qwen2.5-coder:7bqwen3-coder:30bDataset:
Command used:
Docker connectivity to Ollama was verified with:
Result
Before this change:
After this change:
The reward remained
0.0, but the benchmark infrastructure and Codex agent execution completed successfully. This suggests the issue was the generated Codex config format, not Docker, Ollama connectivity, or the verifier.Summary by CodeRabbit