diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 11bb679..b621aea 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -9,7 +9,7 @@ "name": "dtwo", "source": "./dtwo", "description": "DTwo MCP gateway management — bundles the DTwo MCP server and skills for managing gateway configs, deploys, and policies", - "version": "1.0.2", + "version": "1.0.3", "category": "security", "keywords": [ "agentic", diff --git a/dtwo/.claude-plugin/plugin.json b/dtwo/.claude-plugin/plugin.json index 3616378..d7de6fc 100644 --- a/dtwo/.claude-plugin/plugin.json +++ b/dtwo/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dtwo", - "version": "1.0.2", + "version": "1.0.3", "description": "Manage DTwo gateways, policies, and Rego with the DTwo MCP server.", "author": { "name": "DTwo", diff --git a/dtwo/README.md b/dtwo/README.md index 768a2c9..4dbd36f 100644 --- a/dtwo/README.md +++ b/dtwo/README.md @@ -40,8 +40,8 @@ Naming the connector `dtwo` is required — it has to match the plugin's MCP ser | Skill | Use when | | --------------------- | -------------------------------------------------------------------------------------- | | `dtwo-gateway-config` | Editing gateway YAML, adding/removing MCP servers, publishing or rolling back configs. | -| `dtwo-gateway-policy` | Creating, attaching, publishing, deploying, or verifying policies and pipelines. | -| `dtwo-policy-rego` | Authoring, modifying, explaining, or debugging Rego policy code for the DTwo Gateway. | +| `dtwo-gateway-policy` | Creating, attaching, publishing, deploying, or verifying policies and pipelines; managing markers (and the intent registry when those tools are enabled). | +| `dtwo-policy-rego` | Authoring, modifying, explaining, or debugging Rego policy code for the DTwo Gateway, including marker writer/reader policies. | The skills load each other on demand via Claude Code's `Skill` tool — most real tasks pull in two or three together. diff --git a/dtwo/skills/dtwo-gateway-policy/SKILL.md b/dtwo/skills/dtwo-gateway-policy/SKILL.md index 457488b..32656cd 100644 --- a/dtwo/skills/dtwo-gateway-policy/SKILL.md +++ b/dtwo/skills/dtwo-gateway-policy/SKILL.md @@ -4,10 +4,13 @@ description: | Create, validate, attach, publish, deploy, verify, and roll back DTwo policies and their pipeline attachments. This is the system-of-record skill: it owns the DTwo MCP tools for policy lifecycle (create, update, publish, revert) and pipeline lifecycle (attach, deploy, verify), and is responsible for confirming - pipeline attachments before and after deployment. + pipeline attachments before and after deployment. Also manages the session-state marker registry, and + the intent registry when the intent tools are enabled. TRIGGER when: user says "create/add a policy", "modify/update a policy", "block/allow/redact something", "attach/detach policy", "set/update pipeline", "publish/pin policy version", "deploy gateway" after a - policy change. Always pair with dtwo-policy-rego for the Rego authoring/modification step. + policy change; also "register/create/manage a marker", "marker registry", or (only when the intent tools + are enabled) "intent capture", "intent registry", "intent transitions", "intent/marker compatibility". + Always pair with dtwo-policy-rego for the Rego authoring/modification step. SKIP when: task is purely *explaining* existing Rego with no save/attach/deploy intent (use dtwo-policy-rego alone); task is editing gateway YAML or MCP server entries (use dtwo-gateway-config). --- @@ -25,6 +28,41 @@ This skill is typically used alongside others. Invoke them via the `Skill` tool - **dtwo-policy-rego** — load at the start of any task that requires writing, modifying, or explaining Rego. Almost always load this together with `dtwo-gateway-policy` unless the task is pure pipeline attachment of an already-authored policy. - **dtwo-gateway-config** — load when the task also involves editing gateway YAML or adding/removing MCP server entries. +## Core Concepts + +Before choosing an approach, understand how the pieces relate. From smallest to largest: + +- **Policy** — a single unit of Rego that makes one decision about one tool call: allow it, deny it, transform the request/response, and/or write a marker. Policies are authored with the companion `dtwo-policy-rego` skill and stored as records with a draft plus published versions. **A policy on its own is inert** — it does nothing until it is attached to a gateway and deployed. +- **Pipeline** — the ordered list of policies attached to a gateway in one direction. Each gateway has two: an **ingress** pipeline that runs *before* a tool call reaches the upstream server (inspect arguments, identity; block or rewrite the request), and an **egress** pipeline that runs *after* the tool returns (inspect the response; block or redact it). Steps run in array order, and an earlier deny short-circuits later steps — so ordering matters. +- **Gateway** — the runtime that fronts one or more upstream MCP servers and enforces its pipelines. Attaching or editing policies changes only stored state; a **deploy** is what makes the current pipelines live. +- **Marker** — a session-state flag one policy writes and another reads, letting policies coordinate *across* tool calls, directions, and upstream servers within a session (e.g. "PII was seen in an earlier response → block outbound sends now"). A marker is defined once in the registry, then used by a writer policy and a reader policy. See Managing Markers. +- **Intent** *(only when the intent tools are enabled)* — a declared session *purpose* captured into session state and gated on by policies. Built on the same session-state mechanism as markers, with its own registry. See Intent Capture, including its availability gate. + +**Mental model:** *policies* are the decision logic; the *pipeline* is where and when they run; the *gateway* is what enforces them once deployed; *markers* and *intent* are how policies share context beyond a single call. + +**Which skill does what:** this skill (`dtwo-gateway-policy`) owns the lifecycle and orchestration — policy records, pipelines, marker/intent registries, deploy, and verify. The companion `dtwo-policy-rego` skill owns the Rego logic *inside* a policy. Most authoring tasks use both. + +## Choosing the Right Approach + +Match the user's goal to the **smallest** mechanism that solves it, then follow that section: + +| The user wants to… | Use | Where | +|---|---|---| +| Block, allow, or restrict a tool call based on the call itself (tool name, arguments, caller identity) | One **ingress policy** (deny) | Creating a New Policy | +| Block or redact a response based on its content | One **egress policy** (deny or transform) | Creating a New Policy | +| Rewrite a tool's arguments before it runs (e.g. force a filter) | One **ingress transform policy** | Creating a New Policy + `dtwo-policy-rego` | +| Make a decision that depends on something earlier in the session (a prior tool, a prior response, another upstream server) | A **marker** — a writer policy stamps it, a reader policy gates on it | Managing Markers | +| Gate tools on *what the agent is currently doing* | **Intent** *(only if the intent tools are enabled — otherwise not available; do not offer it)* | Intent Capture | +| Turn a policy on/off, reorder it, or pin a version at runtime | **Pipeline attachment** + deploy | Pipeline Attachment | +| Stop a policy's runtime effect, or remove it entirely | **Detach** + redeploy (then `dtwo-delete-policy` only if the record should also go) | Deleting a Policy | + +Guidance: + +- **Prefer one policy.** If a single call carries everything needed to decide, a single ingress or egress policy is the answer — don't reach for markers. +- **Markers are for cross-call state**, not for anything decidable from the current call alone. They add a second policy and a registry entry, so use them only when the decision genuinely depends on earlier session activity. +- **Intent is for session-purpose gating** and is only available when the intent tools are present. If they are not, solve the request with policies and markers and do not mention intent. +- **Compose small policies over one large one** — the companion `dtwo-policy-rego` skill explains why (single-concern policies are easier to test, order, and debug). + ## Prerequisites This skill requires the DTwo MCP server to be connected (`dtwo-*` tools must be loaded). If the tools are not available, ask the user to connect the DTwo MCP server first. @@ -60,34 +98,63 @@ The tools listed below reflect the initial set. The DTwo MCP server may add new | `dtwo-get-policy` | Fetch a single policy by UID (includes draft Rego code) | | `dtwo-get-policy-versions` | List published versions for a policy | | `dtwo-validate-policy-rego` | Validate Rego code without saving — useful for dry-run checks before committing changes (note: `dtwo-add-policy` and `dtwo-update-policy` also validate automatically) | -| `dtwo-add-policy` | Validate and create a new policy (requires name, description, policy, packageName, direction) | -| `dtwo-update-policy` | Update an existing policy's draft — any field (policy, packageName, name, description, direction, tags). Validates Rego when both policy and packageName are provided | +| `dtwo-add-policy` | Validate and create a new policy (requires name, description, policy, packageName, direction). Optionally pass `writableKeySchema` to declare the session-state keys the policy is authorized to write — required for any policy that emits a marker (see Managing Markers) | +| `dtwo-update-policy` | Update an existing policy's draft — any field (policy, packageName, name, description, direction, tags, `writableKeySchema`). Validates Rego when both policy and packageName are provided. `writableKeySchema` is tri-state: **omit** → leave unchanged; **`null`** → clear the field (policy keeps no writable keys); **`[]`** → set an explicit empty list (also leaves no writable keys — practically the same effect as `null`; use `null` as the reset); **`[...]`** → replace with that list | | `dtwo-publish-policy` | Publish the current draft as a new version | | `dtwo-revert-policy` | Restore a published version back into the draft | +| `dtwo-delete-policy` | Permanently delete a policy by UID. Fails if the policy is still attached to one or more gateways — detach it from every gateway first (see Deleting a Policy). Distinct from `dtwo-revert-policy`, which only restores a prior version | | `dtwo-list-claims` | Return the union of JWT claim names observed across the tenant, plus the issuers seen. Defaults to tenant-wide; pass `gatewayUid` to scope to a single gateway when the user asks. Call this when authoring or modifying identity-aware policies so rules can reference claims that actually exist; skip for policies that don't read `input.subject.claims`. | +### Marker Registry Tools + +Markers are session-state flags that one policy writes and other policies read to gate on (see Managing Markers). These tools are **always registered** on the DTwo MCP server — they do not depend on any feature flag. + +| Tool | Purpose | +|------|---------| +| `dtwo-list-markers` | List markers in the registry (optional filters: `name` for exact FQID, `tag`). Returns the marker *vocabulary*, not which markers are currently active on a session | +| `dtwo-get-marker` | Fetch a single marker by UID | +| `dtwo-create-marker` | Create a customer-tier marker (requires `namespace`, `markerId`, `description`, `minimumTtlSeconds`; optional `tags`). Full key is `marker::`. The tool requires only non-empty `namespace`/`markerId`; character-shape rules are validated server-side, not at the tool boundary. `internal` and `dtwo` namespaces are reserved for platform markers | +| `dtwo-update-marker` | Update mutable fields on a customer-tier marker (`description`, `tags`, `minimumTtlSeconds`) | +| `dtwo-delete-marker` | Delete a customer-tier marker. Platform-managed entries cannot be deleted | + +### Intent Registry Tools (conditional — feature-gated) + +> **Availability gate — read this before surfacing anything about intents.** The intent tools below are only registered when the DTwo MCP server is deployed with `enable_intent_tools: true`. **Marker tools (above) are always available; intent tools are not.** Before mentioning intent capture, intent registries, transitions, or intent/marker compatibility to the user, confirm the relevant `dtwo-*-intent*` tools are actually present in your available tool list. **If they are absent, the server is not configured for intent capture — do not present intent capture, the intent registry, transitions, or compatibility to the user, and do not attempt to call these tools.** Treat this subsection and the "Intent Capture" section below as inert in that case. Markers work fully without intent capture, so continue to use them normally. + +When present, these tools manage the intent vocabulary and the rules that govern it. See the Intent Capture section for the workflow. + +| Tool | Purpose | +|------|---------| +| `dtwo-set-intent` | Declare the current session intent (captured by the gateway's egress capture policy). Resolves against the tool's **built-in** intent vocabulary, not the registry (registry-driven resolution is planned). Not customer-available yet — stays behind `enable_intent_tools` until registry-driven resolution ships | +| `dtwo-list-intents` | List intents in the registry (platform `system=true` entries are read-only; customer-tier entries are tenant-scoped) | +| `dtwo-get-intent` | Fetch a single intent by UID | +| `dtwo-create-intent` / `dtwo-update-intent` / `dtwo-delete-intent` | Manage customer-tier intents in the registry vocabulary | +| `dtwo-list-intent-transitions` / `dtwo-set-intent-transition-mode` / `dtwo-add-intent-transition` / `dtwo-delete-intent-transition` | Govern which intent→intent moves are allowed | +| `dtwo-list-intent-compatibility` / `dtwo-create-intent-compatibility` / `dtwo-delete-intent-compatibility` | Govern which markers block which intents at `set_intent` time. `dtwo-create-intent-compatibility` takes `intentUid` + `excludedMarkerUid` | + ### Pipeline & Gateway Tools | Tool | Purpose | |------|---------| | `dtwo-list-gateways` | List gateways with optional filters (name, status, uid) | | `dtwo-get-gateway` | Fetch a single gateway by UID | +| `dtwo-get-gateway-config` | Fetch the gateway's YAML configuration. Used here **read-only** to discover `mcp_servers[].name` and tool names when authoring policies (see Tool Discovery). Returns the **draft** config, which can diverge from what's deployed — confirm names against the deployed config before relying on them. Editing gateway YAML belongs to the companion `dtwo-gateway-config` skill | | `dtwo-set-gateway-pipelines` | Attach policies to ingress/egress pipelines | | `dtwo-get-gateway-pipelines` | Fetch ingress and egress pipeline steps for a gateway, including policy details | | `dtwo-deploy-gateway` | Queue a deployment for the gateway | | `dtwo-get-gateway-deployments` | List deployment tasks for a gateway | | `dtwo-get-deployment` | Check status of a specific deployment | -### Deletion (not supported via MCP) +### Deleting a Policy -The DTwo MCP server does not expose a `delete-policy` tool. `revert-policy` restores a prior version — it does **not** delete. +`dtwo-delete-policy` performs a **permanent** delete by UID. This is different from `dtwo-revert-policy`, which only restores a prior version into the draft — it does **not** delete. -When the user asks to delete a policy: +The delete **fails if the policy is still attached to any gateway**, so detach it everywhere first: -1. **Detach first** — remove the policy from all pipelines with `dtwo-set-gateway-pipelines` (pass `[]` to clear the relevant direction), then redeploy. A detached policy remains in the policy list but has no runtime effect. -2. **Delete via the DTwo web UI** — the MCP surface does not offer deletion. +1. **Detach first** — remove the policy from all pipelines with `dtwo-set-gateway-pipelines` (pass `[]` to clear the relevant direction, or re-send the direction's steps without this policy), then redeploy each affected gateway. A detached policy remains in the policy list but has no runtime effect. +2. **Delete** — call `dtwo-delete-policy` with the `uid`. If the policy is still attached, the call fails with an error naming the gateways still referencing it. Use those names (or `dtwo-get-gateway-pipelines`) to find the remaining attachments, detach them, redeploy, and retry. -If a `dtwo-delete-policy` tool later appears (see the tool-discovery note under Prerequisites), prefer it over this workaround. +Deletion is irreversible and confirmation-worthy — confirm with the user before calling `dtwo-delete-policy`. If they only want to stop the policy's runtime effect (not remove the record), detaching and redeploying is sufficient; leave the policy in place. ## Identifying the Target Gateway @@ -364,10 +431,137 @@ After publishing, call `dtwo-set-gateway-pipelines` again with `policyVersion: 1 **Do not skip step 14.** Leaving the attachment on the draft does not take effect immediately, but the current draft state will be bundled into the *next* deploy of that gateway, whoever triggers it and whatever the reason. A later `dtwo-update-policy` edit — even an experimental one — will then go live on a deploy that was meant for an unrelated change. Pinning to a published version freezes runtime behavior against future draft edits. +## Managing Markers + +Markers are session-state flags that policies write and later policies read to gate on. They give the gateway a shared, tenant- and user-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers — a marker written during a Slack call is visible during a later Jira call for the same user (until its TTL expires). Use them to compose small single-purpose policies that signal to each other without shared code: a **writer** policy stamps a marker when it observes something (PII in a response, a production resource touched), and a **reader** policy on a different tool/pipeline/server gates on it. + +Marker tools are always available (they do not require `enable_intent_tools`). The full lifecycle — register, author writer + reader, attach, deploy — runs through this skill plus `dtwo-policy-rego` for the Rego. The Rego authoring patterns (emitting `session_writes["marker::"]`, walking `input.context.session.policies` to read, and the `writableKeySchema` gotchas) live in the companion `dtwo-policy-rego` instructions — load that skill for the writer/reader bodies. + +**Start simple — the minimal marker is a boolean flag.** A writer stamps `marker::` when it observes a condition; a reader denies (or transforms) whenever that key is present. Presence *is* the signal — no value semantics needed. That flag pattern (the PII example used throughout this section) is the recommended starting point; reach for value-carrying markers only when a flag won't do. Counters and other read-modify-write markers are possible but more involved — a self-incrementing writer has to read its own prior value and re-emit on every call, which keeps refreshing (pinning) the TTL — so they aren't a good first marker. + +**Know these limits before you design** (full list under Marker constraints today): + +- **No runtime inspection** — no tool reads a session's active markers; you verify behaviorally (see Verifying a marker pipeline). +- **No manual clearing** — a marker lifts only when its TTL expires; there is no unset tool. +- **Tenant + user scope** — marker state persists for a user across reconnects and new sessions until the TTL expires; opening a fresh session does not clear it. + +### Registering a marker + +Register the marker in the vocabulary before any policy references it: + +``` +dtwo-create-marker( + namespace = "acme", + markerId = "pii_detected", + description = "Session received PII in a tool response", + minimumTtlSeconds = 3600 +) +``` + +- The full key is `marker:acme:pii_detected`. Customer markers live under any namespace except the reserved `internal` and `dtwo`. +- **Keep the key simple.** The tool requires only non-empty `namespace`/`markerId`; the character-shape rules are validated server-side (when a writer policy is saved and at deploy), not at this tool boundary. In practice, use lowercase alphanumerics with underscores or hyphens and avoid dots, slashes, and spaces, so the key is accepted everywhere it's referenced (registry entry, `writableKeySchema` name, and `session_writes` key must all match exactly — see Authoring the writer policy). +- `minimumTtlSeconds` is the **intended floor** for a writer policy's `ttlSeconds` — but it isn't enforced yet (see the `ttlSeconds` note under Authoring the writer policy), so keep the two in sync manually. Adjust later with `dtwo-update-marker`. + +### Authoring the writer policy — `writableKeySchema` + +A policy that emits a marker must declare the key in its `writableKeySchema` (on `dtwo-add-policy` / `dtwo-update-policy`), or the gateway drops the write. Each entry is: + +- `name` — the session-state key, matching the registered marker **exactly** (e.g. `marker:acme:pii_detected`). Keys are exact-match and never normalized, so this `name`, the `session_writes` key, and the registered marker FQID must be byte-identical (including case) or the write silently drops. The tool accepts any non-empty string; marker-key *shape* (allowed characters, reserved prefixes) is validated server-side on save/deploy, and registry *existence* is enforced at deploy time — the deploy fails on an unregistered key (see the Deploy-time validator note below). +- `jsonSchema` — a **stringified JSON object** (a JSON Schema) for the value the policy writes. Rejected at the tool boundary if it doesn't parse as a JSON object (arrays and primitives fail). Use it strictly (`additionalProperties: false`, `required` lists) so drift is caught. Add `"x-d2-is-marker": true` for marker keys. +- `ttlSeconds` — per-key TTL. For a marker key this **should** be ≥ the marker's registered `minimumTtlSeconds`, but keep them in sync manually: the floor is **not yet enforced** (no save-time or deploy-time check today), so a lower value currently saves, deploys, and simply expires early. +- `onDrop` — behavior when a write fails the schema: `"drop"` (default) silently drops the write (best-effort markers); `"deny_request"` hard-denies the tool call (use for security-critical writes so bugs surface loudly instead of silently letting the call through). + +``` +dtwo-add-policy( + name = "acme-pii-detector", + direction = "egress", + packageName = "acme.egress.pii_detector", + policy = , + writableKeySchema = [{ + name: "marker:acme:pii_detected", + jsonSchema: "{\"type\":\"object\",\"required\":[\"marked_at\",\"source_action\"],\"properties\":{\"marked_at\":{\"type\":\"integer\",\"minimum\":0},\"source_action\":{\"type\":\"string\",\"minLength\":1}},\"x-d2-is-marker\":true,\"additionalProperties\":false}", + ttlSeconds: 3600, + onDrop: "deny_request" + }] +) +``` + +### Attaching, deploying, and reading + +1. Author the reader policy (walks `input.context.session.policies` for the marker key — see `dtwo-policy-rego`). The reader needs no `writableKeySchema`; it only reads. +2. Attach both with `dtwo-set-gateway-pipelines` — the writer on the direction that observes the signal (often egress), the reader on the direction that gates (often ingress). Preserve existing steps. +3. Deploy with `dtwo-deploy-gateway`. This is a **policy-only deploy** — hot-reloaded, no gateway restart, no MCP client disconnect (see Deploying). + +**Deploy-time validator.** The deploy hard-rejects if any attached policy declares a `writableKeySchema` marker key that isn't in the registry, reporting which key is unregistered. This is separate from the key's structural validation (allowed characters, reserved prefixes), which the backend applies when the policy is saved — the registry-existence check runs at deploy time. Register the marker *before* attaching a policy that writes it. + +### Verifying a marker pipeline + +Markers can't be verified the way a single policy can — there is no tool to read active markers (see Marker constraints today), so verification is **behavioral and order-dependent**: a marker does nothing until its writer fires, and its effect is only visible through the reader's decision. The tenant+user scope (below) is what makes the negative case tricky, so mind it: + +1. **Confirm the deploy and attachment** as for any policy — poll `dtwo-get-deployment` to `completed`, then `dtwo-get-gateway-pipelines` to confirm both the writer and the reader landed with the expected `evalNamespace` and version pins. +2. **Trigger the writer first.** Make the tool call that satisfies the writer's condition (e.g. a response containing PII). This is what stamps the marker — nothing is active until the writer fires. +3. **Then exercise the reader** (as the same user). Confirm the reader's guarded tool now denies (or transforms) as intended. The marker stays active until its TTL expires. +4. **Confirm the negative case with a clean marker.** Either use a **short TTL and wait for it to expire**, or test as a **different user** who hasn't triggered the writer — then the reader's tool should succeed, proving it blocks only when the marker is active. Reopening the session as the *same* user does **not** clear the marker (tenant+user scope), so that is not a valid negative test. + +**Tip — validate with a short TTL.** A production-length TTL (say an hour) makes iterating painful: a marker stamped in one test stays set for that user until it expires and masks the next attempt. During validation, set the writer's `writableKeySchema.ttlSeconds` short (e.g. 30–60s) so it clears on its own between iterations. (Since the floor isn't enforced, a short `ttlSeconds` deploys regardless; set the marker's `minimumTtlSeconds` to match via `dtwo-update-marker` so the registry still reflects intent.) Once validated, raise the writer's `ttlSeconds` to the production length with `dtwo-update-policy` (and `minimumTtlSeconds` to match), then republish/redeploy. + +Watch for these: + +- **Order and identity matter.** Calling the reader before the writer has fired, or as a different user, shows the marker absent and the reader allowing — correct behavior, not a bug. Sequence writer-then-reader (reconnecting as the same user won't reset it). +- **A stale marker can mask a result.** If an earlier call already stamped the marker and its TTL hasn't expired, the reader keeps denying for that user. Negative-test with a short TTL you can wait out (see the tip) or as a different user. +- **`onDrop: "deny_request"` surfaces schema problems as a denied *writer* call.** If the tool that should stamp the marker is itself denied, the written value likely failed its `writableKeySchema` (e.g. a float timestamp against a `type: integer` field — see the `time.now_ns()` gotcha in `dtwo-policy-rego`). Fix the value shape, not the reader. +- **To see the marker directly while debugging,** attach a temporary reader-side debug policy that dumps `input.context.session.policies` in a deny reason — the marker analog of the dump-input technique in `dtwo-policy-rego` (Debugging Policies). Detach it when done. + +### Cleanup order (reverse of setup) + +Skipping a step makes the next deploy fail (a policy still claims to write a marker that no longer exists in the registry). Tear down in reverse: + +1. Update/remove the **writer policy** so it no longer references the marker in `writableKeySchema`; redeploy so the write contract leaves the bundle. +2. Delete any **intent/marker compatibility** rows that reference the marker (only relevant when intent tools are enabled — `dtwo-delete-intent-compatibility`); redeploy. +3. `dtwo-delete-marker` — nothing references it now. (`dtwo-delete-marker` does **not** currently check for policy references, so it can leave the bundle inconsistent if you skip step 1.) + +### Marker constraints today + +- **No "list active markers" tool.** `dtwo-list-markers` returns the registry *vocabulary* (the markers that are defined), not which markers are currently set on a given session. A policy can read active markers at evaluation time via `input.context.session.policies` (that's how reader policies work), but there is no MCP tool to query a session's live marker state on demand. +- **No "clear marker" tool.** Markers lift on their own when their TTL expires; there is no MCP tool to unset one. To recover from a marker that is blocking a user, wait out the TTL — a new session for the same user does **not** clear it (state is scoped to tenant + user, not per connection). +- **Multiple writers land in separate per-writer slots.** If two policies declare and emit the same marker key, each write lands under its own writer UID; readers get "any-writer" semantics by walking `session.policies.*`. Prefer one canonical writer per marker. + +## Intent Capture (conditional — feature-gated) + +> **Availability gate — read this first.** Everything in this section depends on the DTwo MCP server being deployed with `enable_intent_tools: true`, which registers the `dtwo-set-intent` / `dtwo-*-intent*` tools listed under Intent Registry Tools. **Before presenting any of this to the user, confirm those tools are in your available tool list. If they are not, do not surface intent capture, the intent registry, transitions, or intent/marker compatibility — the deployment is not configured for it. Say only that intent capture is not enabled on this server if the user asks; do not walk them through a workflow they cannot run.** Markers (above) are unaffected and remain fully usable. + +Intent capture lets the agent declare *what it's trying to do* (`dtwo-set-intent`), captures that into session state via an egress policy, and lets ingress policies gate downstream tools on the current intent. It builds on the same session-state mechanism as markers. + +**Status.** Intent capture is **not customer-available yet.** The intent tools (including `dtwo-set-intent`) stay behind `enable_intent_tools` and will **not** be ungated until the user-intent registry actually drives resolution — today `set_intent` resolves against a built-in vocabulary, not the registry. The enforcement policies themselves are **platform-managed** (see below); `dtwo-set-intent` may also move to a dedicated server later. + +### The enforcement policies are platform-managed + +Two policies do the enforcement: + +- **Egress capture** — captures the declared intent into session state when `dtwo-set-intent` is invoked, validates it against the registry, normalizes the category, denies disallowed transitions, and denies when a currently-active marker is registered incompatible with the proposed intent (`intent_marker_incompatible`). +- **Intent-required gate** — optional: denies every tool call until an intent has been set (`dtwo-set-intent` itself is always allowed so the agent can declare). + +**These are platform-managed policies — end users do not author, attach, copy, or modify them, and you should not offer to.** They are being moved to automatic injection when intent capture is enabled; the platform owns their bodies and wiring (upstream-server naming, internal UIDs), and their Rego may not be visible to users. If a user asks to write or change intent-capture Rego, decline and point them at the platform-managed feature rather than reconstructing it. The only intent surface users drive is the **registry** — the intent vocabulary, transitions, and marker compatibility (below), when the tools are enabled. + +### Intent/marker compatibility + +If a marker should block switching into a given intent, register a compatibility row so the egress capture denies `set_intent` while that marker is active: + +``` +dtwo-create-intent-compatibility( + intentUid = , + excludedMarkerUid = +) +``` + +Example: once `marker:acme:pii_detected` is set, a `set_intent` to `incident_response` is blocked — the session already touched sensitive data. + +**Set-time enforcement only.** The check runs at `set_intent` time. A marker raised *after* an intent is set does **not** retroactively invalidate the current intent. Markers accumulate; intents are validated at the decision point. Tell users this plainly so they don't design around a symmetric re-check that doesn't exist. + ## Limitations - This skill cannot author or modify Rego policies — see the companion `dtwo-policy-rego` instructions - This skill cannot edit gateway YAML or add/remove MCP server entries — see the companion `dtwo-gateway-config` instructions -- This skill cannot delete a policy via the MCP surface — detach via `dtwo-set-gateway-pipelines`, then delete in the DTwo web UI +- This skill cannot delete a policy that is still attached to a gateway — detach via `dtwo-set-gateway-pipelines` and redeploy first, then delete with `dtwo-delete-policy` (see Deleting a Policy) - This skill cannot evaluate policies outside a deployed gateway — verification requires live tool calls against the running gateway - This skill cannot retrieve runtime evaluation logs or OPA decision history from the MCP surface diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index 423be3f..c925f0f 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -6,10 +6,11 @@ description: | catalog contribution structure. TRIGGER when: user asks to write/modify/explain/debug a Rego policy, says "block/allow/redact/transform" a tool call or response, mentions OPA, package paths, `input.payload`, `default allow`, or pastes Rego - for review; asks to contribute to `dtwoai/policy-store`, create catalog policy files, or update app, - industry, bundle, manifest, or tests files for reusable DTwo policies; also when diagnosing blanket denies - or transform conflicts. Pair with dtwo-gateway-policy whenever the resulting Rego must be saved, attached, - or deployed. + for review; asks to write a marker writer/reader policy, `session_writes`, session state, or (only when the + intent tools are enabled) to recognize/explain the platform-managed intent-capture policies (not user-authored); asks to contribute to `dtwoai/policy-store`, create + catalog policy files, or update app, industry, bundle, manifest, or tests files for reusable DTwo policies; + also when diagnosing blanket denies or transform conflicts. Pair with dtwo-gateway-policy whenever the + resulting Rego must be saved, attached, or deployed. SKIP when: task is policy CRUD or pipeline attachment that does not change Rego (use dtwo-gateway-policy alone); task is general OPA usage outside the MCP Gateway; task is editing gateway YAML (use dtwo-gateway-config). --- @@ -20,6 +21,15 @@ description: | You are a Rego policy expert for the DTwo MCP Gateway. You translate natural language security requirements into valid Rego policies, explain existing policies in plain language, and modify policies based on instructions. +## Where a policy fits + +This skill owns the Rego *inside* a policy — the allow / deny / transform / `session_writes` logic for a single tool call. The surrounding pieces it plugs into (a **pipeline** is the ordered list of policies on a gateway direction; a **gateway** enforces them once deployed; **markers** and **intent** let policies share session state) are owned by the companion `dtwo-gateway-policy` skill — see its Core Concepts and Choosing the Right Approach sections to pick the right mechanism before writing Rego. A quick orientation for what you write here: + +- **Deny / allow** — decide a single call from the call itself (ingress) or its response (egress). +- **Transform** — rewrite request arguments (ingress) or redact response content (egress). +- **Marker write / read** — coordinate across calls: one policy stamps a session-state flag (`session_writes`), another reads it (see Session State & Markers). +- **Intent** *(only when the intent tools are enabled)* — session-purpose capture and gating (see Intent-capture policies, including its availability gate). + ## Companion skills This skill is typically used alongside others. Invoke them via the `Skill` tool when relevant (in other agents, use your host's equivalent skill-loading mechanism): @@ -227,6 +237,7 @@ transform := { | `reasons` | `set` | Collects human-readable denial messages. Multiple reasons can fire. | | `reason` | `string` | Joins `reasons` into a single semicolon-delimited string. | | `transform` | `object` | Transformation instructions (redaction, payload replacement). | +| `session_writes` | `object` | Optional. Session-state keys this policy writes (e.g. markers: `session_writes["marker::"] := {...}`). Each written key must be declared in the policy's `writableKeySchema`. See Session State & Markers. | ### Transform Object Fields @@ -332,7 +343,7 @@ Older fields (`input.user`, `input.kind`, `input.payload.name`) are **deprecated }, "payload": {}, // Hook-specific data (see Payload by Hook Type) "tool_metadata": {} | null, // Tool definition (name, url, auth_type, gateway_id, input_schema, ...) - "context": {}, // Mirror of top-level fields (correlation_id, payload, tool_metadata, ...) + "context": {}, // Mirror of top-level fields (correlation_id, payload, tool_metadata, ...); additionally carries context.session.policies[][] (context-only, no top-level equivalent) for reading markers / session state — see Session State & Markers "correlation_id": "", // Request trace ID "request_ip": "", // Client IP address or "unknown" "headers": {}, // Filtered HTTP headers (dict) @@ -922,6 +933,126 @@ reasons contains reason if { - When the aggregator ANDs multiple step policies (`allow if { policy_a.allow; policy_b.allow }`), **all** must evaluate to `true`. If any step policy's package path is wrong, its `allow` is `undefined`, and the AND fails. - Transform-only step policies should use `default allow := true` so they don't block requests when their transform doesn't apply. +## Session State & Markers + +Beyond allow/deny/transform, a policy can **write to session state** and other policies can **read it** — giving the gateway a shared, tenant- and user-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers. The main use is **markers**: session-state flags one policy stamps and another gates on. A marker written during a Slack egress call is visible during a later Jira ingress call for the same user, because markers are scoped by tenant and user (not by server) and persist until their TTL expires. + +This lets you compose small single-purpose policies that signal to each other without shared code, instead of one giant policy that observes everything: + +- **Writer policy** — inspects the current call (arguments, response text, identity, whatever) and stamps a marker when a condition is met. +- **Reader policy** — checks whether a marker is set and allows/denies/transforms accordingly. The writer and reader can attach to different tools, different directions, and different upstream servers. + +Start with the simplest shape — a **boolean flag** whose *presence* is the whole signal (the PII example below). Value-carrying markers (counters, structured payloads) are more advanced and rarely needed; a self-incrementing counter, for instance, has to read its own prior value and re-emit every call, which keeps refreshing the TTL. + +Registering the marker vocabulary, attaching the `writableKeySchema`, and deploying are lifecycle operations owned by the companion `dtwo-gateway-policy` instructions (Managing Markers). This section covers only the **Rego**. + +### Writing a marker (`session_writes`) + +A writer emits `session_writes["marker::"] := ` when its trigger fires. The canonical shape: + +- **The value is stored verbatim — there is no envelope.** The object you assign *is* what's persisted for that key. The `writableKeySchema` entry's `jsonSchema` validates that object **at its top level** — it describes the value object directly, not a nested `value`/`data`/`payload` wrapper. So your object's keys must be exactly the properties the schema declares (typically a couple of flat fields like `marked_at` + `source_action`). +- **The key must be declared** in the policy's `writableKeySchema`, or the gateway drops the write. +- **TTL comes from the schema, never the value.** The entry's `ttlSeconds` is applied by the gateway as the key's expiry when the write lands; it is metadata attached to the key at runtime, not a field of the value, and there is no way to set expiry from within the Rego. + +```rego +package acme.egress.pii_detector + +import future.keywords.if +import future.keywords.in + +default allow := true # writer only observes and stamps; it does not deny + +_email_pattern := `[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}` + +_pii_email_found if { + some text in input.payload.text + is_string(text) + regex.match(_email_pattern, text) +} + +# Stamp the marker when PII is observed in the response +session_writes["marker:acme:pii_detected"] := marker_value if { + _pii_email_found + marker_value := { + "marked_at": time.now_ns(), + "source_action": input.resource.name, + } +} +``` + +A writer is usually a `default allow := true` policy — it observes and stamps, it does not block. (A single policy *can* both deny and write, but prefer separate concerns.) + +**The write only happens when the trigger matches.** The `if { ... }` body gates the write like any other rule, so it uses the same request-matching as the rest of this skill: compare tool names case-insensitively (`lower(input.resource.name) == "-"`, with the server-name prefix), read args via `object.get(input.payload.args, ...)`, and inspect response content via `input.payload.text` on egress. A writer whose trigger never matches (wrong case, missing server prefix, wrong direction) stamps nothing — and there's no error, just an absent marker. Confirm the exact tool name and payload shape with the dump-input technique (see Debugging Policies) before relying on the trigger. + +**Silent-drop trap.** Because the schema is strict (`additionalProperties: false`), slipping any undeclared field into the value fails validation — most commonly a `ttl`/`ttlSeconds`, which belongs on the schema (per the rule above), not the value. With the default `onDrop: "drop"` that failure is **silent**: the marker never lands and readers never see it. If a marker mysteriously isn't being read, check the written value against the schema first. + +```rego +# WRONG — `ttl_seconds` is not a declared schema field, so the entire write silently drops +session_writes["marker:acme:jira_access"] := { + "marked_at": time.now_ns(), + "source_action": input.resource.name, + "ttl_seconds": 3600, # ← remove; TTL belongs on the writableKeySchema, not the value +} + +# RIGHT — value carries only the declared fields; TTL is configured on writableKeySchema +session_writes["marker:acme:jira_access"] := { + "marked_at": time.now_ns(), + "source_action": input.resource.name, +} +``` + +### Reading a marker + +Active session state is exposed to a policy at `input.context.session.policies`, shaped as `policies[][] = ` — an outer object keyed by the UID of the policy that wrote each key, and under each writer the keys it wrote (a marker key `marker::` maps to the object the writer emitted). A marker is stored under the **writer policy's UID**, so the read walks all writer slots and treats the key as truthy if any writer set it: + +```rego +package acme.ingress.pii_gate + +import future.keywords.if + +default allow := true + +_pii_active if { + some writer_uid + input.context.session.policies[writer_uid]["marker:acme:pii_detected"] +} + +# Block outbound Slack sends once PII was seen anywhere in this session +allow := false if { + lower(input.resource.name) == "slack-mcp-slack-send-message" + _pii_active +} + +reason := "PII was detected earlier in this session; outbound Slack sends are blocked. Wait for the marker TTL to expire." if { + lower(input.resource.name) == "slack-mcp-slack-send-message" + _pii_active +} +``` + +- The walk-all-writers pattern (`some writer_uid; input.context.session.policies[writer_uid][key]`) is "present under *any* writer is truthy." To trust only a specific writer, filter on `writer_uid == ""`. +- Deny reasons are user-visible — explain what to do about the block (e.g. "wait for the marker TTL to expire"). Avoid "start a new session": marker state is scoped to tenant + user and survives reconnecting, so a new session for the same user won't clear it. + +### `writableKeySchema` (attached via `dtwo-add-policy` / `dtwo-update-policy`) + +The Rego emits the write; the `writableKeySchema` on the policy record tells the gateway what shape the write must have. It is set through the lifecycle tools (see `dtwo-gateway-policy`), not inside the Rego, but the Rego author owns getting the value shape right. Each entry has `name` (the marker key, matching the registry exactly), `jsonSchema` (a stringified JSON object — a JSON Schema — for the value), `ttlSeconds` (should be ≥ the marker's registered `minimumTtlSeconds` — not enforced yet, so keep them in sync manually), and `onDrop` (`"drop"` — silently drop a schema-failing write; `"deny_request"` — hard-deny the tool call). + +Marker-key *shape* is validated server-side (the backend on save, and at deploy), not at the MCP tool boundary — so keep the key simple (lowercase alphanumerics with `_`/`-`; avoid dots, slashes, spaces) and reference it identically everywhere; keys are exact-match and never normalized (see `dtwo-gateway-policy` → Managing Markers). The `jsonSchema`, though, *is* checked at the tool boundary — it must parse as a JSON object. + +### Marker Rego gotchas + +- **`time.now_ns()` must stay an integer.** Use `time.now_ns()` raw for timestamp fields typed `integer` in the schema. Dividing in Rego (e.g. `time.now_ns() / 1000000`) produces a **float**, which fails a `"type": "integer"` schema — and with `onDrop: "deny_request"` that silently-authored bug will block the tool call. +- **Match the key exactly.** The `session_writes` key, the `writableKeySchema` `name`, and the registered marker FQID must all be the identical `marker::` string. A mismatch drops the write. +- **Multiple writers land in separate slots.** If two policies declare and emit the same key, each write lands under its own writer UID; the walk-all-writers read finds either. Prefer one canonical writer per marker. +- **Reads fail open on absent state.** If `input.context.session.policies` is missing or the marker was never written, the `_active` helper simply doesn't match — the reader allows. Structure high-sensitivity gates so the *presence* of the marker is what denies, not its absence (that's the intended semantics: no signal → nothing to block). + +## Intent-capture policies (conditional — feature-gated) + +> **Availability gate — read this first.** The intent-capture surface (the `dtwo-set-intent` tool and the intent registry) only exists when the DTwo MCP server is deployed with `enable_intent_tools: true`. **Do not present intent-capture policies, `set_intent`, or intent/marker compatibility to the user unless those tools are actually available** — check for `dtwo-set-intent` / `dtwo-*-intent*` in your tool list, or confirm via the companion `dtwo-gateway-policy` instructions. If they are absent, this section is inert; markers (above) still work fully. + +**The intent-capture Rego is platform-managed — do not write or modify it, and do not offer to.** Two policies do the enforcement — an **egress capture** that records the declared intent into session state and denies disallowed transitions or intent/marker incompatibilities, and an optional **intent-required gate** that blocks tool calls until an intent is set. These are owned by the platform (being moved to automatic injection when intent capture is enabled); their bodies and wiring are not user-authored, and the Rego may not be visible to users. If asked to author or change intent-capture Rego, decline and point the user at the platform-managed feature (and the user-facing registry/compatibility tools in `dtwo-gateway-policy` → Intent Capture). This section exists only so you can *recognize and explain* the behavior, not reproduce it. + +One behavior worth knowing when explaining them: intent/marker compatibility is **one-directional and set-time only** — the check runs at `set_intent` time; a marker raised *after* an intent is set does not retroactively deny. + ## Handling Parse and Access Failures Rego rules fail **silently** — if any expression in a rule body fails (e.g., `json.unmarshal` on non-JSON text, or accessing a missing key), the entire rule body does not match. No error is raised. This means the policy's behavior on bad data depends on how the rules are structured.