feat(toolboxes): support skills on toolbox versions by hund030 · Pull Request #8396 · Azure/azure-dev

hund030 · 2026-05-27T09:04:10Z

Summary

Adds skills support on toolbox versions, per the new optional field on ToolboxVersionObject / CreateToolboxVersionRequest in the Foundry spec. Follow-up to #8322.

Changes

internal/pkg/azure: Skills []map[string]any on CreateToolboxVersionRequest (omitempty) and ToolboxVersionObject (always serialized, matching service behavior). []map[string]any keeps future ToolboxSkill discriminator variants forward-compatible.
internal/cmd/toolbox_skill.go: name validator (mirrors azure.ai.skills' regex, documented), parseSkillFlag, buildSkillEntry, validateNoDuplicateSkills.
toolbox create --from-file: optional skills[] block in the input file.
toolbox skill add | remove | list: new subgroup. Each mutation publishes a new immutable toolbox version via createToolboxVersion; no dedicated REST endpoint exists. skill remove allows removing the last skill (skills are optional content, unlike connections).
connection add / connection remove: carry skills forward verbatim into the new version.
toolbox show / toolbox version list: surface skill count and a per-skill table / skills_count column.

Validation

Unit tests cover the new helpers, the create skills-from-file path, carry-forward on connection add/remove, and all three skill verbs.
Live e2e against a Skills-enabled Foundry project: create with skill → connection peel preserves skill → skill remove → empty list → skill add republishes. Full Test_CLI_Toolbox_Smoke flow green.

Adds the skills field to ToolboxVersionObject and CreateToolboxVersionRequest, plumbed through: - toolbox create --from-file: accepts an optional skills[] block. - toolbox skill add/remove/list: post-creation mutation subgroup (no dedicated REST endpoint; each mutation publishes a new immutable version via createToolboxVersion). skill remove allows removing the last skill (skills are optional content). - connection add/remove: carry forward existing skills verbatim into new versions. - toolbox show / toolbox version list: surface skill count + per-skill table.

Copilot

Pull request overview

Adds “skills” support to the azure.ai.toolboxes extension so toolbox versions can reference Foundry Skills (create-time via --from-file, plus new toolbox skill add/remove/list verbs), and surfaces skill counts/details in listing/show commands.

Changes:

Extends Foundry toolbox version request/response models to include skills.
Adds azd ai toolbox skill {add,remove,list} commands with local validation + tests.
Propagates skills across connection add/remove version publishing; updates show and version list output to include skill info.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
cli/azd/extensions/azure.ai.toolboxes/internal/pkg/azure/foundry_toolsets_client.go	Adds `Skills` to toolbox version create request + version response models.
cli/azd/extensions/azure.ai.toolboxes/internal/exterrors/codes.go	Adds new validation/error codes for skill operations.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_version_list.go	Adds `skills_count` to JSON output and SKILLS column to table output.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill.go	New helpers for parsing/validating skill specs and building wire entries.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_verbs_test.go	New tests covering skill add/remove/list flows and edge cases.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_test.go	Unit tests for skill parsing, entry building, and duplicate detection.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_remove.go	Implements `toolbox skill remove` (publishes new version + promotes default).
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_list.go	Implements `toolbox skill list` and row extraction for output.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_group.go	Adds `toolbox skill` command group and shared list helpers.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_add.go	Implements `toolbox skill add` (publishes new version + promotes default).
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_show.go	Surfaces skill count and renders a skills table in `toolbox show`.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_help.go	Updates `--from-file` shape docs to include `skills`.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_files.go	Extends create-file shape with `skills`; improves parse error suggestions.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_create.go	Accepts `skills[]` from create file and validates duplicates.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_connection_remove.go	Carries skills forward when publishing a version during connection removal.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_connection_add.go	Carries skills forward when publishing a version during connection addition.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_commands_test.go	Adds tests for skills-from-file and skills carry-forward semantics.
cli/azd/extensions/azure.ai.toolboxes/internal/cmd/root.go	Registers the new `toolbox skill` command group.

Replace strings.IndexByte/slicing with strings.Cut in parseSkillFlag, and string-append loops with strings.Builder in the over/exactly-64-chars tests. CI enforces these via 'go fix ./...'.

@2

- parseSkillFlag now trims the name portion after splitting on '@' so input like 'my-skill @2' yields Name:'my-skill' instead of leaving a trailing space (which would break duplicate / remove lookups). - runSkillRemoveWith normalizes skillName with TrimSpace at the top, mirroring validateSkillName's internal trim so a user input of ' beta ' matches the canonical stored entry. - toolbox show skills table reuses extractSkillRows so malformed entries are skipped consistently with skill list. - toolbox create file parse-error hint points users at 'skill add' / 'skill remove' instead of implying create-time is the only way to attach skills.

github-actions · 2026-05-27T09:31:53Z

📋 Prioritization Note

Thanks for the contribution! The linked issue isn't in the current milestone yet.
Review may take a bit longer — reach out to @rajeshkamal5050 or @kristenwomack if you'd like to discuss prioritization.

trangevi

Blocking this PR while I get some clarity on the API surface

Addresses two open review comments on PR #8396: - therealjohn: drop the 'in v1' phrasing from user-facing strings; it's not a CLI versioning concern, it's a service-side constraint. - trangevi: support batching on mutation verbs so users don't churn versions one item at a time: - toolbox connection remove: variadic positionals (one or many connection names). - toolbox skill add: gains a --from-file mode (skills[] block) alongside the single positional. - toolbox skill remove: variadic positionals. Each invocation publishes exactly one new toolbox version covering the whole batch. Also tightens internal docs to drop a few TypeSpec-name leaks (buildSkillEntry, the parse-error suggestion text) so the verb files don't reach for terms users won't recognize.

… publish Addresses trangevi's review on PR #8396: toolboxes are shared resources, so a connection-add or skill-mutation should not silently redirect every other consumer to the new version. The PATCH that promotes a version is now a deliberate user gesture via the new toolbox publish verb. - connection add, connection remove, skill add, skill remove: drop the SetDefaultVersion call after publishing a new toolbox version. - Rename toolbox update <name> --default-version <v> to toolbox publish <name> <version> (positional shape matches trangevi's wording). - Mutation success messages now say 'Published toolbox <name> version <v>' and append 'The default version is unchanged; run azd ai toolbox publish ... to promote.' - All error suggestions and help text that mentioned 'toolbox update --default-version' now point at 'toolbox publish'. - Drop two error codes that are no longer reachable (CodeMissingUpdateField, CodeSetDefaultVersionFailed). - Tests: invert setDefaultCalls length-1 assertions to assert.Empty, replace runToolboxUpdate test with the equivalent for runToolboxPublish, update the delete-suggestion substring check.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 10 comments.

…ename to publish Reverts only the verb rename from PR review comment #4. The behavior change (no auto-promote default on mutation verbs) and the help/error/success-message rewording stay. Rationale: toolbox update --default-version <v> is a superset of a hypothetical publish <v> (extensible to future patchable fields). It also parallels azure.ai.skills' skill update --set-default-version, which keeps the two extensions visually consistent. Behavior unchanged from previous commit: - connection add/remove, skill add/remove still publish new versions without promoting. - All mutation success messages still tell the user the default is unchanged and point at the explicit promote step (now spelled 'azd ai toolbox update <name> --default-version <v>'). - Help text on root.go, connection group, skill group, connection add, skill add all reflect 'default unchanged' semantics. Restores: toolbox_update.go, runToolboxUpdate, toolboxUpdateFlags, CodeMissingUpdateField, TestRunToolboxUpdate_MissingDefaultVersion. Removes: toolbox_publish.go and its newToolboxPublishCommand.

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (2)

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_help.go:61

The help text points users to azd ai agent connection list, but the connections command is azd ai connection list (and other suggestions in this extension already use that). Update the referenced command so users aren’t sent to a non-existent subcommand.

  skills          Optional. Existing project skills to attach by reference.
                  Each entry needs 'name'; 'version' is optional (omit to
                  follow the skill's default version).

Project connections must already exist on the Foundry project; this command
does not create them. Run 'azd ai agent connection list' to see available
connections.`

cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_help.go:100

The help text points users to azd ai agent connection list, but the connections command is azd ai connection list (and other suggestions in this extension already use that). Update the referenced command so users aren’t sent to a non-existent subcommand.

The toolbox's existing description is carried forward unchanged; the
description is set at create time and cannot be changed later.

Project connections must already exist on the Foundry project; this command
does not create them. Run 'azd ai agent connection list' to see available
connections.`

- parseSkillFlag / validateNoDuplicateSkills error messages and suggestions no longer reference a non-existent --skill flag (skills are positional or in skills[] of --from-file). - runConnectionRemoveWith now trims whitespace from each connection name (parity with skill remove). User input ' foo ' now matches the stored entry. - Root, connection group, and skill group help text now show the full 'azd ai toolbox update <toolbox> --default-version <version>' form so it's copy-pastable.

Signed-off-by: trangevi <trangevi@microsoft.com>

…oolbox. `update` can be added back if toolboxes ever become mutable. This provides consistency with other UX experience Signed-off-by: trangevi <trangevi@microsoft.com>

Signed-off-by: trangevi <trangevi@microsoft.com>

trangevi · 2026-05-28T20:42:02Z

/check-enforcer override

Copilot AI review requested due to automatic review settings May 27, 2026 09:04

hund030 requested review from JeffreyCA, therealjohn, trangevi and trrwilson as code owners May 27, 2026 09:04

Copilot started reviewing on behalf of hund030 May 27, 2026 09:04 View session

microsoft-github-policy-service Bot assigned hund030 May 27, 2026

Copilot AI reviewed May 27, 2026

View reviewed changes

fix(toolboxes): apply go fix modernizations

bc14222

Replace strings.IndexByte/slicing with strings.Cut in parseSkillFlag, and string-append loops with strings.Builder in the over/exactly-64-chars tests. CI enforces these via 'go fix ./...'.

github-actions Bot added the ext-agents azure.ai.{agents,connections,inspector,projects,routines,skills,toolboxes} extensions label May 27, 2026

hund030 linked an issue May 27, 2026 that may be closed by this pull request

Support attaching Foundry Skills to toolbox versions in azd ai toolbox #8397

Closed

therealjohn requested changes May 27, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_files.go Outdated

trangevi requested changes May 27, 2026

View reviewed changes

viswabalaji approved these changes May 27, 2026

View reviewed changes

trangevi requested changes May 27, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_group.go Outdated

Comment thread cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_add.go Outdated

Comment thread cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_add.go Outdated

hund030 added 2 commits May 28, 2026 16:53

hund030 requested review from Copilot, therealjohn and trangevi May 28, 2026 09:04

Copilot started reviewing on behalf of hund030 May 28, 2026 09:04 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

hund030 requested a review from Copilot May 28, 2026 09:56

Copilot started reviewing on behalf of hund030 May 28, 2026 09:56 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

trangevi approved these changes May 28, 2026

View reviewed changes

Merge branch 'main' into zhihuan/feat-toolbox-skills

0515165

Signed-off-by: trangevi <trangevi@microsoft.com>

github-actions Bot mentioned this pull request May 28, 2026

[aw] Extension PR Labeler failed #8436

Open

Change update to publish for modifying the default version of a t…

fad4224

…oolbox. `update` can be added back if toolboxes ever become mutable. This provides consistency with other UX experience Signed-off-by: trangevi <trangevi@microsoft.com>

therealjohn approved these changes May 28, 2026

View reviewed changes

Linter

fe061e7

Signed-off-by: trangevi <trangevi@microsoft.com>

therealjohn approved these changes May 28, 2026

View reviewed changes

trangevi enabled auto-merge (squash) May 28, 2026 20:42

trangevi merged commit b05df4b into main May 28, 2026
26 of 27 checks passed

hund030 deleted the zhihuan/feat-toolbox-skills branch June 3, 2026 07:48

Conversation

hund030 commented May 27, 2026

Summary

Changes

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📋 Prioritization Note

Uh oh!

Uh oh!

trangevi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

trangevi commented May 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions Bot commented May 27, 2026 •

edited

Loading