Tool policy: write-capable ability categories should be opt-in-by-default in non-interactive runtime modes (pipeline/system)

[chubes4]

Summary

WP_Agent_Tool_Policy provides the mechanism for gating tools (allow_only, deny, categories, mandatory, and runtime opt-in by tool/category), but it ships no safe default for write-capable ability categories in non-interactive runtime modes (pipeline, and by extension any future system-style automation mode). As a result, every consumer must remember to configure an allow-list or opt-in policy; if they don't, write-capable abilities (create/update/publish content) are exposed ambiently to automation agents. This is an opt-out posture for dangerous tools, which is the wrong default for the substrate that's a WordPress Core candidate.

Motivating incident (production)

On a Data Machine install, event-import pipelines (event_import → ai → upsert) had AI steps with an empty tool policy. Because generic content-publishing abilities (publish-wordpress, upsert-post, insert-content) declare pipeline mode and no allow-list was set, the policy resolved to the full pipeline tool pool. The pipeline agent — when it correctly identified a source item as non-music — improvised by calling a generic publish ability to write itself a "rejection report" as a plain post, bypassing the intended upsert_event handler and its domain guards. Result: 224 junk posts silently published across ~75 flows, discovered only via spam comments generating moderation emails.

The consumer-side emergency fix (make generic publish opt-in for pipeline steps) shipped in Data Machine. But the durable home for the default is here in the substrate — no consumer should be able to expose "publish anything" to an automation agent by omission.

Where the gap is (code)

• src/Tools/class-wp-agent-tool-policy.php resolve() — mode/allow_only/deny/category filtering + collect_runtime_tool_opt_in(). The runtime opt-in path (~lines 282–312) handles caller-injected runtime tools, but there is no notion of "this ability category is write-capable and therefore opt-in-by-default in non-interactive modes."
• The raw Abilities API (wp-includes/abilities-api.php) correctly carries only capability + category, not tool-exposure policy — so this does NOT belong there. It belongs in the mediation layer (agents-api).

Proposed change

Introduce a substrate-level safe default: write-capable ability categories are opt-in (not ambient) in non-interactive runtime modes.

Concretely, one or more of:

1. A registerable notion of "write/mutating" category classification (or honor a write: true-style flag on ability-tool declarations), such that these are excluded from the default resolved set in pipeline/system modes unless explicitly opted in via allow_only, runtime_tools, runtime_categories, or an ALLOW-mode policy naming them.
2. Interactive modes (chat) are request-user-scoped and already gated by access checks, so the default there can remain permissive — the tightening targets automation modes specifically.
3. Adjacent/mandatory plumbing tools (the flow's declared next-step handler, completion tools) remain available — this is about ambient generic write tools, not the tools a workflow is explicitly wired to use.

This makes the substrate safe-by-default: a consumer that hands an empty policy gets read + disposition + explicitly-wired handler tools, but NOT "publish arbitrary content," unless it opts in.

Why this matters for Core alignment

WP_Agent_Tool_Policy / wp_register_agent() are being evaluated for WordPress Core alignment. If the substrate ships to millions of sites with an opt-out posture for content-mutating tools in automation contexts, the failure mode demonstrated above (silent unbounded publishing by omission) becomes a platform-wide footgun. Safe-by-default is the correct posture for a Core substrate.

Related

• Consumer-side emergency fix: fix: make generic content-publishing tools opt-in for pipeline AI steps Extra-Chill/data-machine#2853 (generic publish opt-in for pipeline AI steps) — shipped, but re-implements at the consumer what the substrate should default. If this lands, that consumer gate can be simplified to a category declaration.