feat: safe-by-default tool curation — write-capable tools opt-in for autonomous principals#411
Merged
Merged
Conversation
…odes Add a safe-default gate to WP_Agent_Tool_Policy that excludes write-capable tools in non-interactive runtime modes (pipeline, system) unless they are mandatory plumbing or explicitly opted in. Classification mechanism (layer-pure): - Per-tool declaration: 'write' => true or 'mutating' => true on the tool/ability-tool definition. The tool declares its own nature; the policy enforces the default. - Category fallback: a tool whose category matches a write-capable category slug (write, publishing, mutating, destructive by default) is also gated. Exemptions (unchanged behavior): - Interactive chat mode is not affected. - Mandatory tools/categories and declared-mandatory plumbing survive. - Explicit opt-in via allow_only, runtime_tools, runtime_categories, or an ALLOW-mode named policy restores write tools. Escape hatch: - Context or policy fragment key 'allow_write_tools' => true disables the gate entirely for that resolution. - The 'agents_api_write_capable_categories' filter lets Core and consumers extend or replace the built-in category set. The gate reuses the existing collect_runtime_tool_opt_in() aggregate so every opt-in path works uniformly. No vendor/consumer tool slugs are hardcoded — classification is by declared nature or generic category, never by tool name. Addresses Automattic#408.
The tool policy is a neutral substrate and must not enumerate consumer-specific automation mode names. Remove the layer-purity violation introduced by RUNTIME_PIPELINE/RUNTIME_SYSTEM and the NON_INTERACTIVE_MODES membership list: - Delete RUNTIME_PIPELINE and RUNTIME_SYSTEM constants (consumer orchestration vocabulary the substrate defined but never acted on). - Delete the NON_INTERACTIVE_MODES enumeration added in this PR. - Replace is_non_interactive_mode($mode) with a descriptive is_non_interactive_context($context): non-interactive = NOT interactive, reusing the same interactive detection the consent policy already uses (interactive => true OR mode in chat/interactive/rest). - Keep RUNTIME_CHAT = 'chat' (the substrate's own interactive vocabulary). - Rewrite the conversation-store $context docstrings to describe modes as consumer-defined opaque strings with interactive/non-interactive as the substrate's axis. - Rewrite the write-capable smoke test to drive the gate via the interactive axis rather than nuked constants, including cases for an opaque consumer mode string, interactive => true/false, and the interactive markers. The substrate is now fully mode-name neutral. Consumers keep their own product-shaped mode vocabulary and map onto the interactive axis at the call site. Data Machine (the primary consumer) needs zero changes: its chat path already passes interactive => true, and all of its ability categories are datamachine-* namespaced, so none match the substrate's generic write-capable category set (write/publishing/mutating/destructive) and no tool sets write/mutating flags.
Reposition the write-capable gate to key off the execution principal's autonomy instead of a mode/interactive string. The substrate already models autonomy precisely (WP_Agent_Execution_Principal auth_source / acting_user_id), so a mode string was a weaker proxy. A principal is autonomous when its auth_source is an automation source (system / runtime / agent_token) OR it has no human backing it (acting_user_id === 0). For autonomous principals, write-capable tools remain opt-in (curation); human-backed principals are unchanged. The principal is read from a `principal` entry (instance or array shape) or flat auth_source/acting_user_id on the context root. With no principal information present, the gate falls back safe (autonomous = engages). Drop the INTERACTIVE_MODES mode-marker list and the is_non_interactive_context mode check entirely so the gate names NO consumer modes at all. RUNTIME_CHAT is retained as the generic tool mode-filter default (a substrate-native interactive surface the gate no longer touches). Document the gate explicitly as TOOL-SURFACE CURATION (defense-in-depth), NOT the enforcement boundary. Capability-ceiling enforcement at ability/tool execution is tracked separately in Automattic#412. Rewrite tests/write-capable-default-smoke.php to drive the gate via principal autonomy (autonomous principal -> write tools curated out; user/interactive principal -> retained; explicit opt-in -> retained; no-principal -> safe fallback engages; mode string does not move the gate). Update tests/tool-policy-contracts-smoke.php section [5] to model an interactive user principal so the generic runtime-tool opt-in behavior stays in focus without the write curation gate interfering. Part of Automattic#412.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses #408. Part of #412.
Makes write-capable / mutating ability categories opt-in (not ambient) for autonomous principals (no human in the loop), while leaving human-backed (interactive) principals unchanged. This is tool-surface CURATION (defense-in-depth) — it reduces prompt surface and stops an autonomous agent from even trying a write tool. It is NOT the security boundary: the real boundary is capability-ceiling ENFORCEMENT at ability/tool execution, tracked separately in #412.
Production incident this prevents: a pipeline agent silently published 224 junk posts by calling a generic publish ability it was never meant to reach. (Consumer-side emergency fix: Extra-Chill/data-machine#2853; this PR is the durable substrate curation layer.)
Update (this push) — repositioned onto principal autonomy
The gate now keys off the execution principal's autonomy, which the substrate already models precisely, instead of a mode/interactive string (a weaker proxy). Per the deeper analysis in #412, the gate is also now explicitly documented as curation, not enforcement.
WP_Agent_Tool_Policy::INTERACTIVE_MODES(thechat/interactive/restmode-marker list) and theis_non_interactive_context()mode check entirely. The write-gate now names zero consumer modes.is_non_interactive_context($context)withis_autonomous_principal_context($context), keyed off the principal.RUNTIME_CHAT = 'chat'— it is still the default value for the generic toolmodefilter (filter_by_mode(), a substrate-native interactive surface that lets tools declare the modes they are available in). The write-gate no longer touches it.How autonomy is determined
A principal is autonomous when EITHER:
auth_sourceis an automation source (system/runtime/agent_token), ORacting_user_id=== 0).The principal is resolved, in order, from:
principalentry on the context — aWP_Agent_Execution_Principalinstance or its array shape (to_array()), orauth_source/acting_user_idfields on the context root (mirrors howWP_Agent_Action_Policy_Resolveralready readsacting_user_id).When no principal information is present at all, the gate falls back safe: autonomy is assumed and the gate engages. Prefer the principal at the call site; the fallback exists so an unannotated context cannot silently expose write tools.
Why autonomy, not mode
The substrate already models who/what is driving the run via the principal. A mode string only describes the runtime surface (
chat/rest/cli/…), not whether a human is accountable for the loop. Two examples the old mode-based gate got wrong:acting_user_id— a mode-based gate that treatsrestas "interactive" would surface write tools to it.Keying off autonomy fixes both.
git grep -nE "INTERACTIVE_MODES|'pipeline'|'system'|RUNTIME_PIPELINE|RUNTIME_SYSTEM|NON_INTERACTIVE" src/Tools/now returns only theauth_sourceallow-list ('system', 'runtime', 'agent_token').What's excluded by default (for autonomous principals)
Tools classified as write-capable are removed unless preserved or explicitly opted in. A tool is write-capable when either:
'write' => trueor'mutating' => trueon the tool/ability-tool definition. (Layer-pure: the tool declares its own nature, the policy enforces the default. Mirrors the existingaction_policyper-tool declaration precedent.)['write', 'publishing', 'mutating', 'destructive'], all generic classification vocabulary (not consumer tool names).The substrate never hardcodes consumer tool slugs (e.g.
publish-wordpress) and never enumerates consumer mode names. Classification is by declared nature or generic category.What's exempt (unchanged behavior)
auth_sourceuser/application_password withacting_user_id> 0)$preserve_toolclosure still exempts them — mandatory plumbing survives.'mandatory' => true)allow_onlyruntime_toolsruntime_categoriesmode: allow, tools: [...])The gate reuses the existing
collect_runtime_tool_opt_in()aggregate, so every opt-in path works uniformly — no parallel opt-in path was built.Escape hatches
'allow_write_tools' => true'allow_write_tools' => trueagents_api_write_capable_categoriesThe final
agents_api_resolved_toolsfilter still runs after the write gate, so host-level overrides compose on top.Where it sits in
resolve()The gate runs after the runtime opt-in gate (so it can reuse the aggregated opt-in set) and before deny (so explicit deny always wins) and before the final filter (so hosts can still override the result).
Smoke test
tests/write-capable-default-smoke.php(37 assertions) proves the gate keys off principal autonomy, not a mode string:INTERACTIVE_MODES/NON_INTERACTIVE_MODES/RUNTIME_PIPELINE/RUNTIME_SYSTEMare gone;RUNTIME_CHATretained (as the generic mode-filter default); filter methods present.write/mutatingflags gated exactly like category-classified tools.agent_tokenauth_source (even with an acting user) andacting_user_id=0(even for a non-automation auth source).chatmode; user principal keeps write tools even inpipelinemode.allow_only/runtime_categories/ ALLOW policy /runtime_tools→ write tool restored for an autonomous principal.mandatory_tools→ write tool preserved even for an autonomous principal.allow_write_tools(context + policy) → ambient write tools restored.agents_api_write_capable_categoriesfilter → extended category gated for autonomous principal, visible for user principal.agents_api_resolved_toolsfinal filter still runs after the gate.tests/tool-policy-contracts-smoke.phpsection [5] was updated to model an interactive user principal so the generic runtime-tool opt-in behavior stays in focus without the write curation gate interfering.Full suite:
composer smoke(all existing tests still green) andcomposer phpstan(clean).Consumer impact: Data Machine needs zero changes
Data Machine (the primary consumer) requires no code changes for this repositioning. The curation gate is a no-op for DM's tools: every DM ability category is
datamachine-*namespaced (datamachine-publishing,datamachine-content, …) and no DM tool sets awrite/mutatingflag. None of DM's tools match the substrate's generic write-capable set (write/publishing/mutating/destructive). DM's own pipeline write-gating (filterPipelineWriteOptInTools, which targetsrequires_pipeline_opt_in+datamachine-publishing) continues to own product-specific gating exactly as before. The substrate gate simply composes as defense-in-depth without dropping anything DM currently exposes.When DM (or any consumer) wants the substrate to engage for an autonomous principal, it passes the execution principal into the resolver context — the substrate no longer guesses from a mode string.
Relationship to #412
Shipping curation as if it were the security boundary would give Core sites a false sense of safety. This PR is explicitly scoped as curation, with enforcement tracked in #412.
Notes
INTERACTIVE_MODESwas private and never imported by consumers; the only behavioral change is that the gate now engages on principal autonomy (with a safe no-principal fallback) instead of on a mode-string heuristic.