Skip to content

Ship v1: OpenAPI surface, 5 endpoint groups, audit dashboard, Try-It, Discovery#5

Merged
cjimti merged 15 commits into
mainfrom
ship
May 11, 2026
Merged

Ship v1: OpenAPI surface, 5 endpoint groups, audit dashboard, Try-It, Discovery#5
cjimti merged 15 commits into
mainfrom
ship

Conversation

@cjimti
Copy link
Copy Markdown
Contributor

@cjimti cjimti commented May 11, 2026

Summary

Completes the v1 surface api-test was designed for: the Plexara API gateway can now register api-test as a connection via the served OpenAPI spec, exercise five new endpoint groups (streaming/pagination/methods/security/export), and a portal operator gets the dashboard widgets, audit replay, Try-It panel, and Discovery view promised by the docs.

Before this PR, api-test v1.0.0 documented these surfaces as "M3+/M4+ future work." With this merged, every M3+/M4+ marker in the tree is either delivered or removed.

What's new (gateway-facing)

  • OpenAPI 3.1 surfacepkg/oapi generates the spec at boot by reflecting over the registry's EndpointMeta. Served at GET /openapi.json, GET /openapi.yaml, and GET /docs (Redoc viewer). Boot-time oapi.SelfCheck fails startup if the mux and the doc disagree. This is what makes api_list_endpoints work against api-test.
  • Streaming group/v1/streaming/{chunked,sse,ndjson} with deterministic content and context-aware cancellation.
  • Pagination group/v1/pagination/{link,odata,cursor}. Same synthetic dataset across all three styles so the gateway can verify Link header / @odata.nextLink / opaque cursor preservation produce identical items at the same offset. Includes an integer-overflow regression on the link path (caught by adversarial review).
  • Methods group/v1/method/echo verb matrix (GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS) so the gateway can be asserted not to rewrite verbs.
  • Security group — five inert probes shaped to look like dangerous targets (admin/secret, fetch?url=, big-headers, redirect-to?url=, control-chars?q=). The redirect probe uses X-Would-Redirect-To, not Location, so CodeQL's go/unvalidated-url-redirection stays clean.
  • Export group/v1/export/{big-body,csv,long-running} for stressing body-size limits and slow-first-byte handling. CSV writer uses encoding/csv to satisfy gosec G705 on seed taint flow.

What's new (portal-facing)

  • Audit dashboard endpointsaudit/timeseries, audit/breakdown, audit/stats, audit/stream (SSE live tail with within-tick paging + saturated diagnostic frame on tied-timestamp bursts), audit/export.ndjson (offset-stable under concurrent writes via pinned-To window). Adversarial review caught both the silent-event-loss and offset-duplicate bugs before merge.
  • Audit replayPOST /audit/replay/{id} reconstructs and re-dispatches a captured request. Audit middleware reads audit.ReplayHeaderName into Payload.ReplayedFrom, so replays show up linked back to their source. Bounded response buffer via capResponseWriter (no 32 MiB allocation on /v1/sized replays). Feature flag gated on PayloadLogger capability so the SPA doesn't advertise a button that 404s.
  • Try-It dispatchPOST /portal/tryit/{group}/{route} lets a portal operator construct a request from the endpoint catalog. Strips Cookie/Authorization from operator-supplied headers; refuses / in path params (segment-injection guard).
  • SPA Try-It panel — replaces the "arrives in M4" placeholder on the endpoint detail page. Form keyed by endpoint name so state doesn't bleed across endpoints. Shares the replayTarget dispatch field with the replay endpoint.
  • SPA Discovery page — new /portal/discovery route iframes /docs, HEAD-probes /openapi.json for an empty-state fallback. Sandboxed iframe (allow-scripts allow-same-origin).

What's also in here

  • Docs cleanup — all (M3+) / (M4+) / "lands in M3" markers across docs/, configs/, README stripped or rewritten now that those features ship.
  • Config-fixture load testTestShippedFixtureConfigsLoad globs configs/api-test.*.yaml and asserts Load() succeeds. Closes the gap that let a duplicate-YAML-key merge bug squeeze through make verify until an adversarial reviewer caught it manually.
  • AuditMeta.features.stats TypeScript type field — backend serves it; SPA type now declares it.

Stats

  • 69 files changed, +7128 / -112
  • ~700 new test cases across 13 new packages/files
  • Every adversarial pre-commit review returned Verdict: CLEAN. Real bugs caught and fixed during review: integer overflow on pagination's link page param, silent event loss on audit stream bursts, offset-pagination duplicate rows in NDJSON export under concurrent writes, unbounded buffering on replay, replay lineage not actually populated, YAML duplicate-key merge regression, redirect-to handler tripping CodeQL.

Test plan

  • make verify passes against the merged tree (it does locally — fmt, vet, lint, gosec, govulncheck, CodeQL, integration with testcontainers Postgres, UI typecheck+build, coverage gate, mod-tidy/verify)
  • CI runs same on the PR
  • Smoke: make dev boots, curl http://localhost:8080/openapi.json returns a 3.1.0 doc listing all 8 enabled endpoint groups
  • Smoke: register api-test as a Plexara API gateway connection using examples/plexara-connection.yamlapi_list_endpoints reads the spec and enumerates routes
  • Smoke: hit each new endpoint group from a Plexara client; verify entries land in audit_events with the right endpoint_group and route_name
  • Portal smoke: sign in, dashboard shows timeseries/breakdown/stats; SSE stream live-tails new events; NDJSON export downloads; click an event → Replay button; pick a route → Try-It form sends and shows the response
  • Open /portal/discovery — Redoc renders the same spec served at /openapi.json

How this was built

Originally 13 separate branches off main, each individually reviewed CLEAN. Combined into this single ship branch via sequential squash-merges in dependency order. The merge-resolution conflicts (mostly the auditMeta.features map and the per-endpoint-group toggle yaml lists) were resolved by union; each merge commit got its own adversarial sub-agent review which caught two more real bugs (YAML duplicate keys, dispatchTarget/replayTarget name collision) before this PR.

Recommend Squash and merge on GitHub.

cjimti added 15 commits May 10, 2026 17:39
…-check

pkg/oapi walks the endpoint registry's EndpointMeta (PathParams,
QueryParams, RequestBody, ResponseBody typed-nil pointers already
populated by every group) and emits an OpenAPI 3.1 document. JSON tags
drive property naming; ",omitempty" drives required fields; path params
are extracted from "{name}" segments and typed via PathParams when set.

httpsrv mounts the rendered JSON, YAML, and a Redoc viewer at /openapi.json,
/openapi.yaml, /docs. The root banner now advertises them. BuildMux gains
an *oapi.Document parameter; nil disables the surface (tests pass nil).

oapi.SelfCheck runs at boot from internal/server and fails startup if the
served document and the registry disagree, so the doc cannot drift from
the mux. This is the gateway-facing surface the Plexara API gateway uses
to register api-test as a connection (api_list_endpoints reads it).
pkg/endpoints/streaming exposes three deterministic streaming
endpoints so a gateway test can verify the gateway preserves transfer
encoding, flush boundaries, and content type across the proxy hop:

  GET /v1/streaming/chunked  → text/plain, Transfer-Encoding chunked
  GET /v1/streaming/sse      → text/event-stream
  GET /v1/streaming/ndjson   → application/x-ndjson

All three share count, delay_ms, seed query params. Content per item
is a stable function of (seed, index) so callers can replay a stream
and bit-compare. Bounds: count ≤ 1000, delay_ms ≤ 5000.

Each inter-item delay uses a select against the request context so a
client disconnect frees the goroutine immediately instead of running
out the worst-case 83 minutes (count=1000 × delay_ms=5000).

Wired into buildRegistry behind cfg.Endpoints.Streaming.Enabled, on
in both dev.yaml and live.yaml. The OpenAPI generator (on
oapi-generator branch) picks up the new routes via reflection on
QueryParams / ResponseBody automatically once the branches merge.
Closes two gaps surfaced during the merge-resolution review:

1. make verify never loaded configs/api-test.*.yaml, so a duplicate
   YAML key (introduced by the security-merge auto-resolve script) was
   only caught by an adversarial reviewer reading the diff. New
   TestShippedFixtureConfigsLoad in pkg/config globs the directory
   and asserts Load() succeeds on every fixture. yaml.v3 rejects
   duplicate map keys at parse time, so this catches that exact
   class of merge bug going forward. Sets APITEST_INSECURE defensively
   so the test doesn't silently break if a fixture later flips
   skip_signature_verification: true.

2. AuditMeta.features in ui/src/lib/api.ts was missing the stats
   field that the audit-aggregations backend serves. Harmless today
   (TypeScript tolerates the extra key at runtime) but a future
   SPA "is stats enabled?" check would silently always read
   undefined. Adds stats: boolean to the type so backend and SPA
   stay in lockstep.
@cjimti cjimti merged commit 9c2af07 into main May 11, 2026
7 checks passed
@cjimti cjimti deleted the ship branch May 11, 2026 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant