feat: thread attack objective, judge hint, and business use case through sdk, cli, and mcp#193
Merged
jithin23-kv merged 3 commits intoJul 14, 2026
Conversation
RunConfig.attackObjective already exists in core and is wired up for `opfor run --objective`/`--objective-file`, but the SDK and MCP server had no way to set it — only opfor hunt and the CLI run command could steer attacks with a free-text mission. - SDK: add RunOptions.attackObjective, threaded through buildRunConfig() into the core RunConfig. - MCP: add attack_objective to opfor_setup (baked into the written config) and objective_override to opfor_run (overrides it at run time, same pattern as effort_override/turns_override). Both tools' response text now echo the objective when set. - docs: document the new SDK option and MCP parameters.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughChangesEvaluator steering configuration
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant CLI_or_MCP
participant RunConfig
participant Evaluator
participant PromptBuilder
participant Judge
Operator->>CLI_or_MCP: provide steering fields
CLI_or_MCP->>RunConfig: store or override configuration
RunConfig->>Evaluator: pass objective, context, and judge hint
Evaluator->>PromptBuilder: generate attacker prompt
Evaluator->>Judge: provide combined judge hint
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@runners/mcp/src/index.ts`:
- Around line 231-237: Update the attack_objective and objective_override
schemas to trim input and reject empty or whitespace-only strings. In the setup
flow, preserve valid empty checks by testing both override values with !==
undefined rather than truthiness, so validated objectives are not silently
dropped.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: f39d110c-eded-47ab-8ac3-7c4fb5882bb8
📒 Files selected for processing (10)
core/src/config/schema.tscore/src/execute/evaluatorLoop.tscore/src/execute/types.tsdocs/cli.mddocs/mcp.mddocs/sdk.mdrunners/cli/src/commands/run.tsrunners/mcp/src/index.tsrunners/sdk/src/internal/buildRunConfig.tsrunners/sdk/src/types.ts
jithin23-kv
changed the title
There was a problem hiding this comment.
🧹 Nitpick comments (2)
core/src/generate/generateAttacks.ts (1)
154-166: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winPositional params are getting risky — consider an options object.
generatePatternAgentAttackandbuildAgentSystemPromptnow take several consecutive optionalstring | undefinedparams (traceContext,attackObjective,businessUseCase). The order currently checks out at both call sites, but this shape makes future edits easy to silently misorder since the type checker can't catch swapped same-typed args.♻️ Proposed direction
-async function buildAgentSystemPrompt( - evaluator: EvaluatorSpec, - targetDescription: string, - model: LanguageModel, - traceContext?: string, - upstreamSessions?: SessionContext[], - attackObjective?: string, - businessUseCase?: string -): Promise<string> { +async function buildAgentSystemPrompt( + evaluator: EvaluatorSpec, + targetDescription: string, + model: LanguageModel, + opts?: { + traceContext?: string; + upstreamSessions?: SessionContext[]; + attackObjective?: string; + businessUseCase?: string; + } +): Promise<string> {Also applies to: 200-202
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/src/generate/generateAttacks.ts` around lines 154 - 166, Update generatePatternAgentAttack and buildAgentSystemPrompt to accept a named options object for the consecutive optional context, attackObjective, and businessUseCase values instead of positional parameters. Update both call sites to pass these properties by name, preserving the current values and behavior while preventing same-typed arguments from being silently reordered.core/src/config/schema.ts (1)
166-168: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winConsider bounding these free-text fields.
attackObjective,judgeHint, andbusinessUseCaseare unconstrained strings that get injected verbatim into every attacker/judge prompt (generateAttacks.ts) and are also settable via MCP runtime overrides per the PR objectives. A missing.max(...)means an oversized value silently balloons every generated prompt's token count/cost across the whole run.♻️ Proposed fix
- attackObjective: z.string().optional(), - judgeHint: z.string().optional(), - businessUseCase: z.string().optional(), + attackObjective: z.string().max(4000).optional(), + judgeHint: z.string().max(2000).optional(), + businessUseCase: z.string().max(4000).optional(),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/src/config/schema.ts` around lines 166 - 168, Bound the free-text fields attackObjective, judgeHint, and businessUseCase in the relevant schema definition by adding an appropriate maximum length to each optional string. Preserve their optional behavior and ensure the same validation applies to values supplied through MCP runtime overrides.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@core/src/config/schema.ts`:
- Around line 166-168: Bound the free-text fields attackObjective, judgeHint,
and businessUseCase in the relevant schema definition by adding an appropriate
maximum length to each optional string. Preserve their optional behavior and
ensure the same validation applies to values supplied through MCP runtime
overrides.
In `@core/src/generate/generateAttacks.ts`:
- Around line 154-166: Update generatePatternAgentAttack and
buildAgentSystemPrompt to accept a named options object for the consecutive
optional context, attackObjective, and businessUseCase values instead of
positional parameters. Update both call sites to pass these properties by name,
preserving the current values and behavior while preventing same-typed arguments
from being silently reordered.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3ee7c985-5769-4064-91b2-36866292caf2
📒 Files selected for processing (13)
core/src/config/schema.tscore/src/execute/evaluatorLoop.tscore/src/execute/runAllBrowser.tscore/src/execute/types.tscore/src/generate/generateAttacks.tscore/tests/runAll.smoke.test.tsdocs/cli.mddocs/mcp.mddocs/sdk.mdrunners/cli/src/commands/run.tsrunners/mcp/src/index.tsrunners/sdk/src/internal/buildRunConfig.tsrunners/sdk/src/types.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- runners/mcp/src/index.ts
- docs/cli.md
- docs/mcp.md
- runners/cli/src/commands/run.ts
achuvyas-kv
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
There was no way to steer every evaluator's attacks toward a specific free-text
mission, bias the judge's verdict, or give the attacker business/domain context —
each evaluator just pursued its own generic goal with no operator input.
opfor huntalready had--objectivefor its autonomous mode, and the browserextension's "Advanced" panel exposed three operator-intent fields (Attack
objective, Custom evaluator hint, Business use case), but the evaluator-based
opfor runpath had no equivalent on any other surface (CLI, SDK, or MCP).Solution
Added
attackObjective,judgeHint, andbusinessUseCaseend-to-end across allthree ways of driving
opfor run:RunConfig.attackObjective/.judgeHint/.businessUseCase(schema + type), applied onto every generated agent attack in
runEvaluatorAttacks, and threaded intogenerateAttacks()so comprehensivemode's turn-1 seed prompt sees them too (not just adaptive mode's follow-up
turns).
judgeHintis combined with each attack's existing evaluator-authoredhint rather than overwriting it.
opfor run --objective <text>/--objective-file <path>,--judge-hint <text>/--judge-hint-file <path>,--business-use-case <text>/
--business-use-case-file <path>.RunOptions.attackObjective/.judgeHint/.businessUseCase,threaded through
buildRunConfig().attack_objective/judge_hint/business_use_caseonopfor_setup(baked into the written config) andobjective_override/judge_hint_override/business_use_case_overrideonopfor_run(runtimeoverride, same pattern as
effort_override/turns_override). All three toolsecho the fields in their response text when set.
Changes
core/src/config/schema.ts,core/src/execute/types.ts— new fields/schema onRunConfigcore/src/execute/evaluatorLoop.ts— stampsattackObjective/businessUseCaseonto agent attacks, combinesjudgeHintwith each attack's existing hintcore/src/generate/generateAttacks.ts— threadsattackObjective/businessUseCaseinto the comprehensive-mode turn-1 seed promptcore/src/execute/runAllBrowser.ts— passes the same fields intogenerateAttacksfor the browser path's comprehensive moderunners/cli/src/commands/run.ts—--objective/--objective-file,--judge-hint/--judge-hint-file,--business-use-case/--business-use-case-filerunners/sdk/src/types.ts,runners/sdk/src/internal/buildRunConfig.ts— SDK supportrunners/mcp/src/index.ts— MCP supportcore/tests/runAll.smoke.test.ts— new tests asserting the actual rendered prompt content (attackerATTACK_OBJECTIVE/BUSINESS_CONTEXTblocks, judgeJUDGE HINTcombine behavior) across both adaptive and comprehensive effortdocs/cli.md,docs/sdk.md,docs/mcp.mdIssue
N/A
How to test
npm run buildopfor run --config <file> --objective "<text>" --judge-hint "<text>" --business-use-case "<text>"— confirm the log prints all three lines and generated attacks/judge prompts reflect them.run({ ..., attackObjective, judgeHint, businessUseCase })and confirm they reach the generated attack/judge prompts.opfor_setupwithattack_objective/judge_hint/business_use_caseset, confirm they're written intoopfor.config.json; then callopfor_runwith the matching*_overrideparams and confirm they override the config values (response text shows all three).npm run typecheck && npm run lint && npm run format:check && npm testall pass.Screenshots
N/A — no UI/report surface changes.
Summary by CodeRabbit
--objectiveand--objective-file; MCP setup and run tools support corresponding objective/judge/business overrides.