feat(agent-builder): author, save, and dry-run custom tools#3237
feat(agent-builder): author, save, and dry-run custom tools#3237dmarticus wants to merge 4 commits into
Conversation
Wire the backend's custom-tool authoring loop (posthog#66584) into PostHog Code so an internal user can create, save, and dry-run a custom tool end to end, behind the existing agent-platform feature flag. v0: correctness over polish. API client + types (hand-written; agent-platform types are not generated): - Add ToolCompileError(Kind), ToolCapabilities, WriteToolRequest/Result, DryRunToolRequest/Envelope/Result to @posthog/shared. - Add putRevisionTool (422 -> typed compile-failed result, inline; other non-2xx throw), deleteRevisionTool (404 -> success), and dryRunRevisionTool (reads `ok` from the body; 200 and 500 return the envelope; 429 -> throttled; 503 -> unavailable; never thrown, never retried). A parseFailedRequest helper recovers status + body from the shared fetcher's throw. Hooks: - useSaveRevisionTool (invalidates the bundle only when the compile persisted), useDeleteRevisionTool, useDryRunRevisionTool. All expose isPending. UI (agent builder): - ToolSourceEditor: draft-only editable source with a compile-on-save, inline ToolCompileError diagnostics (kind + message + 1-based line/column), double-submit guard, and a capabilities readout from the save response. - ToolDryRunPanel: args JSON editor, optional mock_secrets rows, Test button, and a result area handling success / tool failure / 429 busy / 503 unavailable. - Gate edit + dry-run behind AGENT_PLATFORM_FLAG; non-draft/non-custom stays read-only. Tests: client cases for 200/422/409 writes, delete 404-as-success, and dry-run 200-ok/200-fail/500/429/503 + mock_secrets passthrough; a hook test for the invalidate-only-when-persisted branch. Note: end-to-end dry-run needs the deployed backend (posthog#66584) with the sandbox configured; until then it returns 503, the expected "not available here" state. Generated-By: PostHog Code Task-Id: 3dc0ab45-e514-4c17-a185-9aa78871a3ba
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
React Doctor found no issues in the changed files. 🎉 Reviewed by React Doctor for commit |
|
…e timer - Make DryRunToolResult.throttled `max_concurrent` optional and omit it on a 429 without a numeric count, so the panel no longer renders a misleading "0 runs in flight". The dry-run panel shows a count only when present. - Clear the "saved" tick setTimeout in ToolSourceEditor on unmount and before the next save (via a ref) so a stale timer can't setState after unmount. - Add a client test for a 429 with no max_concurrent. Generated-By: PostHog Code Task-Id: 3dc0ab45-e514-4c17-a185-9aa78871a3ba
- ToolDryRunPanel: give mock-secret rows a stable id (ref counter) and key/ update/remove by it, instead of the array index. - ToolSourceEditor: make the source textarea uncontrolled (defaultValue + ref) and track only a `dirty` flag, dropping the prop-derived useState and the in-render reconcile. The parent keys the editor by revision+tool so switching either remounts it with a fresh buffer. - Key each compile-error row by its kind/line/column/message instead of the array index. Generated-By: PostHog Code Task-Id: 3dc0ab45-e514-4c17-a185-9aa78871a3ba
The tool detail view showed an always-open source textarea and an always- expanded dry-run panel on draft tools, which made the node look like a code editor bolted into the builder. Extract a ToolSourcePanel that defaults to a collapsed, read-only view: - Resting state is a `source · <path>` header plus a "Show source" toggle (read-only CodeBlock when expanded) — no editor, no open dry-run panel. - "Edit source" (draft-only) is an explicit toggle that swaps in the inline editor; "Test tool" is an explicit toggle that reveals the dry-run panel, with a "Hide test" to collapse it. - Keyed by revision+tool so switching either resets the toggles and reseeds the editor buffer. ToolSourceEditor/ToolDryRunPanel are unchanged — just moved behind the toggles. Generated-By: PostHog Code Task-Id: 3dc0ab45-e514-4c17-a185-9aa78871a3ba
What
Wires the backend's custom-tool authoring loop (posthog#66584) into PostHog Code. Until now the agent builder could only read tool source (
tools/<id>/source.ts, read-only) — this adds the client + a minimal GUI to create, save, and dry-run a custom tool end to end, behind the existingagent-platformflag.This is v0: correctness over polish. No capabilities panel beyond a save-time readout, no monaco intellisense, no per-team quota UX — those are v1.
How
Part A — API client + types (hand-written; agent-platform types are not generated from OpenAPI)
@posthog/shared/agent-platform-types:ToolCompileError/ToolCompileErrorKind,ToolCapabilities,WriteToolRequest/WriteToolResult,DryRunToolRequest/DryRunToolEnvelope/DryRunToolResult.posthog-client.ts:putRevisionTool— 422 returns a typed compile-failed result (rendered inline); other non-2xx (400/409 sealed/…) still throw.deleteRevisionTool— 404 (tool_not_found) treated as success.dryRunRevisionTool— readsokfrom the body (a throwing tool is HTTP 200 withok:false); 200 and 500 both return the envelope; 429 →throttled(withmax_concurrent), 503 →unavailable. Never thrown, never retried.parseFailedRequesthelper recovers status + body from the shared fetcher'sFailed request: [status] bodythrow.Part B — hooks
useSaveRevisionTool(invalidates thebundle(...)query only when the compile persisted),useDeleteRevisionTool(invalidates bundle),useDryRunRevisionTool(no invalidation). All exposeisPending.Part C — UI (agent builder)
ToolSourceEditor— draft-only editable source with compile-on-save, inlineToolCompileErrordiagnostics (kind + message + 1-based line/column), Revert, double-submit guard, and asecret_refs/dynamic_secret_refsreadout from the save response.ToolDryRunPanel— args JSON editor, optionalmock_secretskey/value rows, Test button, and a result area handling success (result +duration_ms), tool failure (error.code/message), 429 busy, and 503 unavailable.ToolBodygates edit + dry-run behindAGENT_PLATFORM_FLAG; non-draft / non-custom tools stay read-only.Testing
pnpm typecheck(23/23 packages) andpnpm biome checkon all touched files pass.mock_secretspassthrough. New hook test for the invalidate-only-when-persisted branch.@posthog/api-client78 pass,@posthog/ui1325 pass.Note
End-to-end dry-run requires the deployed backend (janitor + main app) to include posthog#66584 with the sandbox backend configured. Until then dry-run returns 503, which is the expected "not available here" state this UI is built against.
🤖 Generated with Claude Code