Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions src/cli/operations/eval/__tests__/pause-resume.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -176,4 +176,43 @@ describe('handlePauseResume', () => {
expect(result.error.message).toContain('Could not extract config ID');
});
});

describe('name + ARN cross-validation', () => {
it('accepts matching name and ARN', async () => {
mockLoadDeployedProjectConfig.mockResolvedValue(makeContext('my-config', 'cfg-123'));
mockUpdateOnlineEvalExecutionStatus.mockResolvedValue({
configId: 'cfg-123',
executionStatus: 'DISABLED',
status: 'ACTIVE',
});

const arn = 'arn:aws:bedrock-agentcore:us-east-1:123456789012:online-evaluation-config/cfg-123';
const result = await handlePauseResume({ name: 'my-config', arn }, 'pause');

assert(result.success);
expect(result.executionStatus).toBe('DISABLED');
});

it('rejects mismatched name and ARN', async () => {
mockLoadDeployedProjectConfig.mockResolvedValue(makeContext('my-config', 'cfg-123'));

const arn = 'arn:aws:bedrock-agentcore:us-east-1:123456789012:online-evaluation-config/different-cfg';
const result = await handlePauseResume({ name: 'my-config', arn }, 'pause');

assert(!result.success);
expect(result.error.message).toContain('refer to different configs');
expect(mockUpdateOnlineEvalExecutionStatus).not.toHaveBeenCalled();
});

it('returns name-lookup error when both are passed but name is unknown', async () => {
mockLoadDeployedProjectConfig.mockResolvedValue(makeContext('other-config', 'cfg-999'));

const arn = 'arn:aws:bedrock-agentcore:us-east-1:123456789012:online-evaluation-config/cfg-999';
const result = await handlePauseResume({ name: 'missing-config', arn }, 'pause');

assert(!result.success);
expect(result.error.message).toContain('missing-config');
expect(result.error.message).toContain('not found');
});
});
});
25 changes: 24 additions & 1 deletion src/cli/operations/eval/pause-resume.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,34 @@ function parseOnlineEvalConfigArn(
}

/**
* Resolve config ID and region from either a project config name or an ARN.
* Resolve config ID and region from a project config name, an ARN, or both.
*
* When both are provided, the named config is looked up and its configId is
* cross-checked against the ARN. A mismatch means the user passed a name and
* ARN that point to different configs — we reject rather than silently
* preferring one (the ARN previously won, so the name was a no-op).
*/
async function resolveConfig(
options: OnlineEvalActionOptions
): Promise<{ success: true; configId: string; region: string } | { success: false; error: string }> {
if (options.arn && options.name) {
const arnResolution = parseOnlineEvalConfigArn(options.arn, options.region);
if (!arnResolution.success) return arnResolution;

const nameResolution = await resolveOnlineEvalConfig(options.name);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When both --arn and a config name are passed, this branch now calls resolveOnlineEvalConfig(options.name)loadDeployedProjectConfig() unconditionally. But the command layer in src/cli/commands/pause/command.tsx deliberately skips requireProject() whenever --arn is present (lines 38–40 / 129–131):

if (!cliOptions.arn) {
  requireProject();
}

So a user running this outside a project directory with agentcore pause online-insights some-name --arn arn:... will now hit a ConfigNotFoundError from readProjectSpec() instead of either succeeding (pre-PR behavior) or getting a clear validation message. The throw falls through to the outer try/catch in command.tsx and surfaces as a generic Error: agentcore.json not found ..., which is confusing UX.

A few ways to fix:

  1. In command.tsx, always call requireProject() when a name argument is provided, even alongside --arn. That makes the contract explicit: "if you pass a name, you must be in a project."
  2. In resolveConfig here, catch the ConfigNotFoundError (or check for project presence first) and return a clearer error like "Cannot cross-check config name "X" against --arn: not inside a project. Pass only --arn, or run from a project directory."
  3. Make the cross-check best-effort — if the project can't be loaded, silently fall through to the ARN-only path. (Partly defeats the purpose of the fix, so probably not preferred.)

Option (1) seems cleanest and matches user intent.

if (!nameResolution.success) return nameResolution;

if (nameResolution.configId !== arnResolution.configId) {
return {
success: false,
error:
`--arn and config name "${options.name}" refer to different configs ` +
`(name resolves to "${nameResolution.configId}", ARN resolves to "${arnResolution.configId}"). ` +
`Pass only one, or pass matching values.`,
};
}
return arnResolution;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the configIds match, this returns arnResolution, whose region comes from the ARN (or --region override). But nameResolution.region comes from the project's awsTargets entry for the deployed target. If those disagree, that's the same class of silent name-vs-ARN inconsistency this PR is closing on the configId axis — just on the region axis instead.

Consider also comparing nameResolution.region against arnResolution.region and rejecting (or warning) on mismatch. Otherwise the surface area of "silent disagreement between name and ARN" isn't fully closed.

}
if (options.arn) {
return parseOnlineEvalConfigArn(options.arn, options.region);
}
Expand Down
Loading