From 501d11d34053f1aee1c5adf11244b8a2c2a5d3a4 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 16:31:07 +0000 Subject: [PATCH 01/12] docs: add CONTRIBUTING.md and PR template grounded in repo conventions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the two always-loaded per-repo convention files the repo was missing. Both encode this repo's REAL gates (not boilerplate): scripts/validate-full.sh full validation, the seam-awareness / real-E2E discipline, and the documented mode false positive. New contributors/agents inherit the house rules AGENTS.md already carries. Docs-only; full bundle validation (scripts/validate-full.sh) stays PASS in validation_mode: full (convention files are not bundle-structural). πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .github/PULL_REQUEST_TEMPLATE.md | 46 +++++++++++++++++ CONTRIBUTING.md | 89 ++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 .github/PULL_REQUEST_TEMPLATE.md create mode 100644 CONTRIBUTING.md diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 00000000..fe4fb4ac --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,46 @@ + + +## Summary + + + +## Scope / guardrails + + + +## Verification + +- [ ] **Module tests** pass β€” `modules/tool-context-intelligence-query` (paste count) +- [ ] **Top-level tests** pass β€” `tests/` (paste count) +- [ ] **`ruff check` + `ruff format --check`** clean +- [ ] **`pyright`** clean (0 errors) +- [ ] **Full bundle validation** PASS β€” `scripts/validate-full.sh` β†’ `validation_mode: full` + (the lone mode-advertising ERROR is a documented FALSE POSITIVE β€” see AGENTS.md β€” do NOT "fix" it) + +## Real evidence on seams (not mock-only) + + + +- [ ] Seam(s) crossed by this change are proven against real behavior (or `N/A β€” no seam crossed`) + +## Docs & diagrams + +- [ ] `bundle.dot` / `bundle.png` regenerated if bundle structure changed (the validator flags `BUNDLE_DOT_STALE`) +- [ ] README / SKILLs / agent files updated if a tool/skill contract changed +- [ ] Convention files updated if this surfaced a lasting lesson (`AGENTS.md`, this template, etc.) + +## Notes / follow-ups + + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..4eec08d0 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,89 @@ +# Contributing + +Thanks for contributing to **amplifier-bundle-context-intelligence**. This repo has +a few specific conventions β€” read them before you change the furniture. + +**Read [`AGENTS.md`](AGENTS.md) first.** It is the authoritative, always-loaded +guidance: the known validator false positive, the full-validation command, and the +**seam-awareness** rules that govern how changes to tool/skill/config wiring must be +tested. Everything below is the short version. + +## Layout + +- `bundle.md` β€” thin root pointer. `behaviors/` β€” layered, composable behaviors + (`-navigation` βŠ‚ `-analysis` βŠ‚ `-design`, orthogonal `-logging`, umbrella + `context-intelligence`). `agents/`, `skills/`, `modes/`, `context/` β€” bundle content. +- `context_intelligence/` β€” the shared Python library. `modules/` β€” the installable + modules (`tool-context-intelligence-query`, `hook-context-intelligence`, + `tool-context-intelligence-upload`). `tests/` β€” top-level suite. + +## Branches & commits + +- Branch names: `feat/…`, `fix/…`, `docs/…`, `chore/…`. +- Conventional-commit subjects (`feat(query): …`, `fix(client): …`, `docs(skill): …`). +- Keep unrelated changes in separate commits (code / docs / deps). + +## Dev setup + +Uses [`uv`](https://docs.astral.sh/uv/). From a module dir or the repo root: + +```bash +uv sync +``` + +`ruff` and `pyright` are declared as dev dependencies, so they are available in the venv. + +## Testing β€” the gates + +Unit tests are the floor, not the ceiling. Run and paste evidence for: + +```bash +# module + top-level suites +uv run pytest # in modules/tool-context-intelligence-query +uv run pytest # in the repo root (tests/) +uv run ruff check . && uv run ruff format --check . +uv run pyright +``` + +**Real evidence on seams (this repo's signature rule).** Per `AGENTS.md` +"Seam Awareness": a mock that *fabricates the boundary's response* (e.g. a fake +server returning canned rows) is **not a gate** until it has been reconciled to real +behavior. If your change touches a seam β€” the client↔server boundary, blob handling, +tool/skill/config wiring β€” prove the **real** crossing (a live run against a real +server, or a Digital Twin run), not just a passing mock. Live testing here has caught +bugs that green mocks did not. + +## Full bundle validation + +Before opening a PR that touches bundle structure, run the repo's **full** validation +(not the bare recipe, which self-downgrades to `structural_only`): + +```bash +scripts/validate-full.sh +``` + +It builds a throwaway `uv` venv with `hatchling` + `amplifier-foundation` + +`amplifier-core` so the validator runs at `validation_mode: full`. The lone +mode-advertising **ERROR** it reports is a **documented FALSE POSITIVE** (a name +collision β€” see `AGENTS.md`); **do not "fix" it** by advertising the internal mode or +deleting path/skill references. + +If your change altered bundle structure, regenerate the diagram and commit it +(the validator flags `BUNDLE_DOT_STALE` otherwise): + +```bash +# regenerate bundle.dot / bundle.png via the generate-bundle-docs recipe +``` + +## Pull requests + +Open PRs against `main` and **populate every item in the PR template** from real +evidence β€” paste it, or mark `N/A β€” `. Never pre-check a box you cannot back. +State what you deliberately did **not** touch (e.g. the write-side hook fan-out for a +read-side change). + +## Capturing lessons + +If your work surfaces a lasting lesson β€” a footgun, an invariant, a new gate β€” write it +back into the file that owns it (`AGENTS.md` for pitfalls/commands, the PR template for +a new gate) as it lands. Offer the entry and get agreement; keep what's worth keeping. From 0252d5318249877130fdb1b0507a138c853f78c6 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 16:53:27 +0000 Subject: [PATCH 02/12] =?UTF-8?q?docs(agents):=20add=20Testing=20+=20"what?= =?UTF-8?q?=20done=20looks=20like"=20=E2=80=94=20real=20DTU/eval=20evidenc?= =?UTF-8?q?e=20required?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Aligns AGENTS.md with the per-repo-convention that it must carry the test commands that gate "done" and what "done" looks like. Adds the unit/lint/type + full-validation commands, and makes explicit that green unit tests are the FLOOR, not proof: any change that crosses a seam β€” the client↔server boundary, networking, auth, or ANY agent / skill / mode / tool / hook / config edit β€” requires a real DTU run plus the evaluation harness (.amplifier/digital-twin-universe/profiles/ + the eval-design / evaluation-methodology skills), with captured end-to-end evidence of working order. A seam crossed on a green mock is not done until reconciled to real behaviour. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- AGENTS.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 3d0a9747..0113abe5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,6 +47,53 @@ interpreter that has the deps β†’ `validation_mode: full`. green, the lone mode "error" confirmed a false positive (name collision). Only the build *dry-run* is skipped (no `pip wheel` in the venv); the wheels build cleanly under `uv build`. +## Testing & what "done" looks like + +Run these before calling anything done: + +```bash +uv run pytest # in modules/tool-context-intelligence-query (module suite) +uv run pytest # in the repo root (tests/, top-level suite) +uv run ruff check . && uv run ruff format --check . && uv run pyright +scripts/validate-full.sh # β†’ validation_mode: full, overall PASS +``` + +**Green unit tests are the FLOOR, not proof of done.** This bundle wires **skills, modes, +networking, tools, and auth** β€” capabilities whose real behaviour lives at **seams** (see +*Seam Awareness* below), where a passing mock can hide a real break. Real, recent proof: two +read-path bugs (a blob-envelope parse and a `ci-blob://` bare-key normalization) passed *every* +unit test and were caught only against a **live server**. + +So **"done" for any change that touches a seam is not "units are green" β€” it is real, +captured end-to-end evidence that the bundle works in production order.** A seam is any edit at +the edges: skill / mode / tool / hook / config wiring, the client↔server boundary, networking, +or auth. For those changes, done **requires**: + +- **A real end-to-end run β€” Digital Twin Universe (DTU) or an equivalent live run** β€” exercising + the ACTUAL code path a user hits: real server, real network, real auth. Not an inbound mock of + the boundary (which is *banned as a gate* until reconciled to real behaviour β€” see *Seam Awareness*). +- **Captured evidence:** the real request/response, the identity/provenance of what actually + answered, and the **fail-loud** behaviour on a real failure (down / 5xx / timeout / bad auth). +- If you crossed a seam without real evidence, it is **NOT done** β€” reconcile the mock to the real + thing first. + +**Agent, skill, and mode edits ALWAYS require a DTU run + the evaluation harness β€” never ship them +on unit tests alone.** An agent's delegation, a skill's guidance, a mode's gating, and the +tool/networking/auth wiring behind them only prove out when *loaded and exercised in a real +environment*. This repo ships the harnesses for exactly that β€” use them, don't reinvent them: + +- **DTU profiles** β€” `.amplifier/digital-twin-universe/profiles/` (e.g. `ci-bundle-smoke-test.yaml`, + `context-intelligence-redesigned-mode-validation.yaml`, `ci-signals-validation.yaml`). Launch the + change in a DTU and drive the real agent/skill/mode/tool path end-to-end. +- **Evaluation methodology** β€” the `context-intelligence-eval-design` and + `context-intelligence-evaluation-methodology` skills. Design/run the evaluation scenarios that + score the behaviour, and capture the results as the evidence of working order. + +Match the gate to the change: pure-internal logic β†’ units + `validate-full.sh`; **a seam crossing β€” +including ANY agent / skill / mode / tool / networking / auth edit β†’ units + `validate-full.sh` + a +real DTU run + the evaluation harness, with captured evidence.** Skipping the live run on a seam +change is how a green build ships a broken bundle. + ## Architecture note This bundle ships **layered, composable behaviours** β€” `context-intelligence-navigation` βŠ‚ From 7f996baec187178fb028bd2967b6552e28ceac92 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 17:16:54 +0000 Subject: [PATCH 03/12] test(dtu): expand ci-* profile names to context-intelligence-*; add mode-activation profile MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Rename DTU profiles to drop the ambiguous `ci-` prefix (could read as "continuous integration"): ci-bundle-smoke-test β†’ context-intelligence-bundle-smoke-test, ci-signals-validation β†’ context-intelligence-signals-validation (git mv, history preserved; internal name:/--name/usage comments and the AGENTS.md references updated). - Add context-intelligence-mode-activation-validation.yaml: a single-purpose DTU test proving explicit /context-intelligence activation mounts the FULL mode-gated surface β€” both contributes.agents specialists resolve, all THREE contributes.context files inject (closing the previously un-asserted context-intelligence-strategy.md), all THREE skills become discoverable (closing the un-asserted evaluation-methodology skill), and the mode tool policies take effect β€” with a clean offβ†’onβ†’off round-trip and an explicit /context-intelligence slash-command activation scenario. Baseline agents stay present. All 6 profiles parse as valid YAML. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- ...ntext-intelligence-bundle-smoke-test.yaml} | 6 +- ...telligence-mode-activation-validation.yaml | 159 ++++++++++++++++++ ...text-intelligence-signals-validation.yaml} | 8 +- AGENTS.md | 4 +- 4 files changed, 168 insertions(+), 9 deletions(-) rename .amplifier/digital-twin-universe/profiles/{ci-bundle-smoke-test.yaml => context-intelligence-bundle-smoke-test.yaml} (97%) create mode 100644 .amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml rename .amplifier/digital-twin-universe/profiles/{ci-signals-validation.yaml => context-intelligence-signals-validation.yaml} (97%) diff --git a/.amplifier/digital-twin-universe/profiles/ci-bundle-smoke-test.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml similarity index 97% rename from .amplifier/digital-twin-universe/profiles/ci-bundle-smoke-test.yaml rename to .amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml index e218c77e..4af00076 100644 --- a/.amplifier/digital-twin-universe/profiles/ci-bundle-smoke-test.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml @@ -5,9 +5,9 @@ # # Usage: # export GH_TOKEN=$(gh auth token) -# amplifier-digital-twin launch .amplifier/digital-twin-universe/profiles/ci-bundle-smoke-test.yaml \ -# --name ci-bundle-smoke-test -name: ci-bundle-smoke-test +# amplifier-digital-twin launch .amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml \ +# --name context-intelligence-bundle-smoke-test +name: context-intelligence-bundle-smoke-test description: > Smoke test for the context-intelligence bundle. Validates library imports, upload CLI, hook additional_events config, and Amplifier bundle loading. diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml new file mode 100644 index 00000000..77398501 --- /dev/null +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml @@ -0,0 +1,159 @@ +# Context-intelligence mode ACTIVATION β€” DTU validation profile. +# +# PURPOSE +# Prove the explicit `/context-intelligence` activation entry point mounts the +# FULL mode-gated surface and that `/mode off` removes it cleanly: +# - all mode-gated AGENTS resolve (the two contributes.agents specialists) +# - all mode-gated CONTEXT is injected (all THREE contributes.context files) +# - all mode-gated SKILLS become discoverable (all THREE contributes.skills) +# - the mode tool policies take effect +# Baseline agents (graph-analyst, session-navigator) stay present throughout β€” +# they are registered at the behavior level, NOT gated by the mode. +# +# This complements the two existing mode profiles: +# - context-intelligence-redesigned-mode-validation.yaml β†’ the 5-phase workflow end-to-end +# - context-intelligence-contributes-migration-validation.yaml β†’ the contributes.agents gating migration +# Neither of those asserts the THIRD context file (context-intelligence-strategy.md) +# or the THIRD skill (context-intelligence-evaluation-methodology). This profile does β€” +# it is the single-purpose "explicit activation mounts the whole gated surface" test. +# +# ACTIVATION MECHANISM +# The mode's user-facing entry point is the explicit slash command `/context-intelligence` +# (mode name = context-intelligence; advertised: false, so it is hidden from `mode list` +# but activatable by name). Structural assertions below use the harness `when: { mode: ... }` +# key β€” the harness's realization of that activation β€” and the behavioral scenario drives +# activation the way a user does, by sending `/context-intelligence` as the message. +# +# USAGE +# amplifier-digital-twin launch \ +# .amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml \ +# --name context-intelligence-mode-activation-validation \ +# --var gitea_host=http://your-gitea:3000 + +profile: + name: context-intelligence-mode-activation-validation + version: 1.0.0 + description: > + Proves explicit /context-intelligence activation mounts ALL mode-gated agents, + context files, and skills β€” and that /mode off removes them cleanly. Baseline + agents remain present throughout. Closes the strategy-context and + evaluation-methodology-skill coverage holes left by the other two mode profiles. + +# Gitea url_rewrites enable local-branch testing: point {gitea_host} at a Gitea +# instance with your branch mirrored. Two forms are rewritten: +# 1. the bare @main URL (bundle root, used by `amplifier bundle add`) +# 2. the #subdirectory=skills variant (skill companion loading) +url_rewrites: + - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main" + to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main" + - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" + to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" + +install: + # Behaviour-only install: add just the context-intelligence behavior (mode, + # agents, tool policies, contributes.*) to the app-level Amplifier config. + command: "amplifier bundle add git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml --app" + +assertions: + - id: A1-baseline-mode-inactive + description: > + BEFORE activation (mode inactive): the two mode-gated specialists are NOT + registered, the mode-gated context is NOT injected, and the design skills are + NOT discoverable. Baseline agents ARE present. The mode is hidden from listings + (advertised: false). + when: + mode: inactive + expect: + agents_present: + - context-intelligence:graph-analyst + - context-intelligence:session-navigator + agents_absent: + - context-intelligence:context-intelligence-design-facilitator + - context-intelligence:context-intelligence-tool-designer + context_file_not_present: + - "context-intelligence:context/jsonl-event-schema.md" + - "context-intelligence:context/dual-path-library-template.md" + - "context-intelligence:context/context-intelligence-strategy.md" + skills_absent: + - context-intelligence-tool-design + - context-intelligence-eval-design + - context-intelligence-evaluation-methodology + mode_not_listed: context-intelligence + + - id: A2-explicit-activation-mounts-full-gated-surface + description: > + AFTER explicit /context-intelligence activation: BOTH specialists resolve, + ALL THREE mode-gated context files are injected, ALL THREE mode-gated skills + are discoverable, and the mode tool policies are in force. Baseline agents + remain present. + when: + mode: active + expect: + agents_present: + - context-intelligence:graph-analyst + - context-intelligence:session-navigator + agents_available: + - context-intelligence:context-intelligence-design-facilitator + - context-intelligence:context-intelligence-tool-designer + context_files_injected: + - "context-intelligence:context/jsonl-event-schema.md" + - "context-intelligence:context/dual-path-library-template.md" + - "context-intelligence:context/context-intelligence-strategy.md" + skills_present: + - context-intelligence-tool-design + - context-intelligence-eval-design + - context-intelligence-evaluation-methodology + tool_policies: + safe: [graph_query, blob_read, read_file, glob, grep, delegate, load_skill, todo] + warn: [bash, write_file, edit_file] + + - id: A3-mode-off-clean-unmount + description: > + AFTER /mode off (mode was active, then deactivated): the specialists, the + mode-gated context, and the design skills are all gone. Baseline agents and the + general (always-on) skills remain. + when: + mode: inactive + after: activation + expect: + agents_absent: + - context-intelligence:context-intelligence-design-facilitator + - context-intelligence:context-intelligence-tool-designer + agents_present: + - context-intelligence:graph-analyst + - context-intelligence:session-navigator + context_file_not_present: + - "context-intelligence:context/jsonl-event-schema.md" + - "context-intelligence:context/dual-path-library-template.md" + - "context-intelligence:context/context-intelligence-strategy.md" + skills_absent: + - context-intelligence-tool-design + - context-intelligence-eval-design + - context-intelligence-evaluation-methodology + general_skills_present: + - blob-reading + - context-intelligence-graph-query + - context-intelligence-session-navigation + +scenarios: + - id: S1-explicit-slash-command-activates-mode + description: > + Drive activation the way a USER does β€” send the explicit `/context-intelligence` + slash command β€” and confirm the mode actually turns on (transition event) and the + gated specialists become invokable. This proves the real user entry point, not just + the harness's internal mode toggle. + setup: + mode: inactive + user_message: "/context-intelligence" + pass_criteria: + mode_active: context-intelligence + jsonl_event: + type: "mode:transition_completed" + payload_path: "data.mounted" + payload_contains: + - "context-intelligence:context/jsonl-event-schema.md" + - "context-intelligence:context/dual-path-library-template.md" + - "context-intelligence:context/context-intelligence-strategy.md" + agents_available: + - context-intelligence:context-intelligence-design-facilitator + - context-intelligence:context-intelligence-tool-designer diff --git a/.amplifier/digital-twin-universe/profiles/ci-signals-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml similarity index 97% rename from .amplifier/digital-twin-universe/profiles/ci-signals-validation.yaml rename to .amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml index cbf06e98..dc123ebd 100644 --- a/.amplifier/digital-twin-universe/profiles/ci-signals-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml @@ -1,4 +1,4 @@ -# ci-signals-validation.yaml +# context-intelligence-signals-validation.yaml # # End-to-end integration test for context_intelligence.signals library, # CLI commands, and render-findings output. @@ -9,10 +9,10 @@ # Usage: # export GH_TOKEN=$(gh auth token) # amplifier-digital-twin launch \ -# .amplifier/digital-twin-universe/profiles/ci-signals-validation.yaml \ -# --name ci-signals-validation +# .amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml \ +# --name context-intelligence-signals-validation # -name: ci-signals-validation +name: context-intelligence-signals-validation description: > Integration test for the context_intelligence.signals library. Validates signal scoring, CLI commands, and render-findings output diff --git a/AGENTS.md b/AGENTS.md index 0113abe5..d51193a0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -82,8 +82,8 @@ on unit tests alone.** An agent's delegation, a skill's guidance, a mode's gatin tool/networking/auth wiring behind them only prove out when *loaded and exercised in a real environment*. This repo ships the harnesses for exactly that β€” use them, don't reinvent them: -- **DTU profiles** β€” `.amplifier/digital-twin-universe/profiles/` (e.g. `ci-bundle-smoke-test.yaml`, - `context-intelligence-redesigned-mode-validation.yaml`, `ci-signals-validation.yaml`). Launch the +- **DTU profiles** β€” `.amplifier/digital-twin-universe/profiles/` (e.g. `context-intelligence-bundle-smoke-test.yaml`, + `context-intelligence-redesigned-mode-validation.yaml`, `context-intelligence-signals-validation.yaml`). Launch the change in a DTU and drive the real agent/skill/mode/tool path end-to-end. - **Evaluation methodology** β€” the `context-intelligence-eval-design` and `context-intelligence-evaluation-methodology` skills. Design/run the evaluation scenarios that From 0042e5e66b0591cac72349800c93d0f2f5778195 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 17:31:39 +0000 Subject: [PATCH 04/12] =?UTF-8?q?docs(dtu):=20profiles=20README=20?= =?UTF-8?q?=E2=80=94=20inventory=20+=20how=20to=20spin=20up=20a=20CI=20ser?= =?UTF-8?q?ver=20for=20e2e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documents the DTU profile suite for this bundle: what each profile really tests, its prerequisites, and its launch command; and β€” per the AGENTS.md seam/real-evidence rule β€” how to stand up and use a Context-Intelligence server for read-side end-to-end tests (external host server via example-dtu-external-server.yaml, or a Docker sidecar inside the DTU via docker-in-incus), the AMPLIFIER_CONTEXT_INTELLIGENCE_* env wiring, the logβ†’queryβ†’assert flow, and the read-side (server_url, api_key) resolution chain. Records working-order status: prereqs present (amplifier-digital-twin 0.3.0, Incus 7.2, Docker), all profiles valid YAML with valid references. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .../digital-twin-universe/profiles/README.md | 121 ++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 .amplifier/digital-twin-universe/profiles/README.md diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md new file mode 100644 index 00000000..af86bde2 --- /dev/null +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -0,0 +1,121 @@ +# Context-Intelligence DTU Profiles + +Digital Twin Universe (DTU) profiles for testing this bundle in isolated, realistic +environments. Launched with [`amplifier-digital-twin`](https://github.com/microsoft/amplifier-bundle-digital-twin-universe). + +Per this repo's `AGENTS.md` ("Testing & what 'done' looks like"): green unit tests are +the floor, not proof. Any change that crosses a **seam** β€” the client↔server boundary, +networking, auth, or any agent / skill / mode / tool / hook / config edit β€” must be +proven with a **real DTU run**, not a mock. These profiles are that harness. + +## Prerequisites + +| Need | Check | Install | +|------|-------|---------| +| DTU CLI | `which amplifier-digital-twin` | `uv tool install git+https://github.com/microsoft/amplifier-bundle-digital-twin-universe@main` | +| Incus (all profiles) | `incus version` | load the `digital-twin-universe` skill β†’ `docs/installing-incus.md` | +| Docker (server sidecar / Gitea) | `docker version` | load the skill β†’ `docs/installing-docker.md` | +| Gitea (local-branch profiles) | β€” | load the `gitea` skill; mirror this repo, pass `--var gitea_host=...` | +| `ANTHROPIC_API_KEY`, `GH_TOKEN` | env | for any profile that runs a real Amplifier session | + +## Profile inventory (what each really tests) + +| Profile | Tests | Needs | Launch | +|---------|-------|-------|--------| +| `context-intelligence-bundle-smoke-test.yaml` | Hook late-contributor path: a module mounting *after* the hook still lands its event in `events.jsonl` | Incus + `ANTHROPIC_API_KEY` + `GH_TOKEN` | `amplifier-digital-twin launch .amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml` | +| `context-intelligence-signals-validation.yaml` | The deterministic `signals` scoring library + CLI (score fixtures, thresholds, render-findings). **No LLM, no server, no Gitea** | Incus only | `amplifier-digital-twin launch .../context-intelligence-signals-validation.yaml` | +| `context-intelligence-redesigned-mode-validation.yaml` | The 5-phase `/context-intelligence` mode end-to-end (tool policies, context injection, specialists, Phase-0/2 artifacts) | Incus + Gitea mirror + LLM | `... launch .../context-intelligence-redesigned-mode-validation.yaml --var gitea_host=http://:3000` | +| `context-intelligence-contributes-migration-validation.yaml` | The `contributes.agents` gating migration (atomic mount w/ JSONL proof, clean unmount, sub-session delegation, skill search) | Incus + Gitea mirror + LLM | `... launch .../context-intelligence-contributes-migration-validation.yaml --var gitea_host=...` | +| `context-intelligence-mode-activation-validation.yaml` | Explicit `/context-intelligence` activation mounts the FULL gated surface β€” both specialists, all 3 context files, all 3 skills, tool policies β€” with an offβ†’onβ†’off round-trip | Incus + Gitea mirror + LLM | `... launch .../context-intelligence-mode-activation-validation.yaml --var gitea_host=...` | +| `example-dtu-external-server.yaml` | *Not a test* β€” reference profile: point the client hook at an **external CI server** with a tagged workspace | Incus + running CI server (below) | see below | + +**Self-contained smoke to prove the harness works on your host:** +`context-intelligence-signals-validation.yaml` needs only Incus (no LLM/server/Gitea) β€” launch it first to confirm the DTU pipeline is healthy before the heavier profiles. + +--- + +## Spinning up & using a Context-Intelligence server for end-to-end tests + +The **read side** (`graph_query`, `blob_read`) and the **write side** (the telemetry +hook) both talk to a **Context-Intelligence server** β€” a separate component +([`microsoft/amplifier-context-intelligence`](https://github.com/microsoft/amplifier-context-intelligence), +backed by Neo4j, serving `/status`, `/events` (write), `/blobs/{session}` and Cypher +(read) on port `8000`). A true read-side e2e test needs a real server: **log events +via the hook, then query them back** via `graph_query` / `blob_read` and assert the +result + provenance. + +There are two ways to give a DTU a server. + +### Option A β€” External server on the host (simplest; the shipped path) + +Run the server **outside** the DTU (on the host or another machine), and point the +DTU's client at it. This is exactly what `example-dtu-external-server.yaml` does. + +1. **Start the server on the host** (per the server repo β€” Neo4j + server, listening on `:8000`): + ```bash + # in a checkout of microsoft/amplifier-context-intelligence β€” see that repo's README + # (typically a docker compose bringing up Neo4j + the API on :8000) + curl -sf http://localhost:8000/status # confirm it's up + ``` +2. **Export the API key on the host** so the DTU passthrough can forward it: + ```bash + export AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY= + ``` +3. **Launch the DTU pointed at it.** The server URL is auto-detected from the Incus + bridge gateway (host), or set it explicitly: + ```bash + amplifier-digital-twin launch \ + .amplifier/digital-twin-universe/profiles/example-dtu-external-server.yaml \ + --var CONTEXT_INTELLIGENCE_WORKSPACE=e2e-readside \ + --var CONTEXT_INTELLIGENCE_SERVER_URL=http://:8000 # optional; auto-detected if omitted + ``` + The profile wires `AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL / _API_KEY / _WORKSPACE` + into the container and gates readiness on `curl $SERVER_URL/status`. +4. **Drive the e2e flow** inside the DTU (`amplifier-digital-twin exec -- ...`): + - run an Amplifier session β†’ the hook POSTs events to `/events` (tagged with the workspace); + - then read them back with the query tools β€” `graph_query` (Cypher, filtered by + `workspace = "e2e-readside"`) and `blob_read` a `ci-blob://` URI β€” and assert the + rows + the `source` provenance block name the server that answered. + +> **Container DNS/networking:** if the client can't reach the server from inside the +> container, see `docs/container-dns-troubleshooting.md` (the gateway-IP + port pattern). + +### Option B β€” Server as a Docker sidecar *inside* the DTU (self-contained) + +Run the server **and Neo4j** as Docker containers **inside** the DTU via the +docker-in-incus pattern (DTU launches enable `security.nesting=true` by default). Use +this when you want one hermetic environment with no host dependency. + +1. Read the nested-container guide first: load the `digital-twin-universe` skill β†’ + `read_file("@digital-twin-universe:docs/docker-in-incus.md")` (platform-specific + networking, pre-flight with the `docker-in-incus` profile). +2. In the profile's `provision.setup_cmds`, bring up the server's compose (Neo4j + API) + from the server repo, then point the client at `http://localhost:8000` via the same + three `AMPLIFIER_CONTEXT_INTELLIGENCE_*` env vars as Option A. +3. Gate readiness on `curl -sf http://localhost:8000/status`, then run the same + logβ†’queryβ†’assert e2e flow as Option A. + +> The exact compose/run invocation lives in the **server repo** +> (`microsoft/amplifier-context-intelligence`) β€” this bundle is the client side. Pin the +> server version you're testing against and record it in your run evidence. + +### Read-side config the tools resolve (both options) + +`graph_query` / `blob_read` resolve `(server_url, api_key)` per field via +`ToolConfigResolver`: **explicit read-config β†’ hook `destinations` β†’ env +`AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL` / `_API_KEY`**. Configuring the hook +`destinations` alone is enough β€” you don't have to repeat the endpoint for the read +tools (see the main `README.md` Β§"read side"). For multi-source, the connectable set is +`sources βˆͺ destinations`; `list_sources: true` shows it. + +--- + +## Working-order status + +- All profiles parse as **valid YAML** and reference only agents/skills/context that + exist in this bundle. +- Host prerequisites confirmed present: `amplifier-digital-twin` 0.3.0, Incus 7.2, Docker. +- **Runtime-green is per-launch**, per the AGENTS.md rule β€” capture the run evidence + (real request/response, provenance, fail-loud on a down/500/timeout) when you exercise + a seam. Start with `context-intelligence-signals-validation.yaml` (no external deps) to + confirm the harness, then the Gitea-mirror mode profiles for mode/agent/context changes. From 0483a297838cb5a0284d8caa982fda65ff9818c0 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 17:43:38 +0000 Subject: [PATCH 05/12] docs(dtu): record live signals-profile run proof in working-order status MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Launched context-intelligence-signals-validation.yaml (Incus, no external deps): readiness "all checks passed", signals symbols import, all 12 fixtures scored with 0 failures; instance then destroyed. Confirms the DTU pipeline (provision β†’ install β†’ run β†’ assert) is healthy on a standard host β€” real evidence, not static validity. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .amplifier/digital-twin-universe/profiles/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index af86bde2..a2def95c 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -115,6 +115,11 @@ tools (see the main `README.md` Β§"read side"). For multi-source, the connectabl - All profiles parse as **valid YAML** and reference only agents/skills/context that exist in this bundle. - Host prerequisites confirmed present: `amplifier-digital-twin` 0.3.0, Incus 7.2, Docker. +- **Harness proven live:** `context-intelligence-signals-validation.yaml` was launched + (Incus, no external deps) and passed end-to-end β€” readiness `all checks passed`, the + `signals` public symbols import, and **all 12 fixtures scored OK / 0 failures**; + instance then destroyed. This confirms the DTU pipeline (provision β†’ install β†’ run β†’ + assert) is healthy on a standard host. - **Runtime-green is per-launch**, per the AGENTS.md rule β€” capture the run evidence (real request/response, provenance, fail-loud on a down/500/timeout) when you exercise a seam. Start with `context-intelligence-signals-validation.yaml` (no external deps) to From 42b1a32cffa37b2debb3095a59ce2c51276b569a Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 18:32:22 +0000 Subject: [PATCH 06/12] docs(dtu): correct DTU testing principle + document DTU/Gitea dev dependencies MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DTU structural/behavioural verification loads the bundle through the Amplifier CLI (`amplifier bundle add` β†’ activate `/context-intelligence` β†’ drive a real session), NOT by running pytest artefacts in a container (that is a unit test in a different directory, not end-to-end). Documents the extra dev dependencies this requires β€” amplifier-digital-twin CLI, Incus, Docker, Gitea (to serve the local branch so the install resolves to our code), and a real LLM key for behavioural scenarios β€” with concrete install pointers and per-profile needs, in the DTU profiles README, and adds an end-to-end/DTU section to CONTRIBUTING.md pointing devs to it. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .../digital-twin-universe/profiles/README.md | 43 +++++++++++++++---- CONTRIBUTING.md | 15 +++++++ 2 files changed, 49 insertions(+), 9 deletions(-) diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index a2def95c..9c5ba742 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -8,15 +8,40 @@ the floor, not proof. Any change that crosses a **seam** β€” the client↔server networking, auth, or any agent / skill / mode / tool / hook / config edit β€” must be proven with a **real DTU run**, not a mock. These profiles are that harness. -## Prerequisites - -| Need | Check | Install | -|------|-------|---------| -| DTU CLI | `which amplifier-digital-twin` | `uv tool install git+https://github.com/microsoft/amplifier-bundle-digital-twin-universe@main` | -| Incus (all profiles) | `incus version` | load the `digital-twin-universe` skill β†’ `docs/installing-incus.md` | -| Docker (server sidecar / Gitea) | `docker version` | load the skill β†’ `docs/installing-docker.md` | -| Gitea (local-branch profiles) | β€” | load the `gitea` skill; mirror this repo, pass `--var gitea_host=...` | -| `ANTHROPIC_API_KEY`, `GH_TOKEN` | env | for any profile that runs a real Amplifier session | +## How these DTUs verify (the principle) + +**Structural and behavioural checks load the bundle through the Amplifier CLI β€” they do +NOT run code artefacts.** A real DTU test does what a user does: `amplifier bundle add` +the bundle, activate `/context-intelligence`, and drive the tools / agents / skills / mode +through an actual `amplifier` session, then assert the *loaded* behaviour. Running +`pytest` (or importing modules) inside a container is **not** an end-to-end test β€” it's a +unit test in a different directory. The unit/integration suites already run via +`uv run pytest` and `scripts/validate-full.sh`; DTUs exist to prove the *installed, +CLI-loaded* bundle actually works. + +Consequence: any profile that verifies loading or behaviour **installs via +`amplifier bundle add`** (see the mode profiles' `install.command`) and, for local-branch +work, resolves that install to your branch through **Gitea** (`url_rewrites` + +`--var gitea_host=...`). + +## Dev dependencies & setup + +To run these DTUs during development you need the following on your host. Add them once: + +| Dependency | Why | Check | Add it | +|------------|-----|-------|--------| +| **`amplifier-digital-twin` CLI** | launches/manages DTU environments | `amplifier-digital-twin --version` | `uv tool install git+https://github.com/microsoft/amplifier-bundle-digital-twin-universe@main` | +| **Incus** | the container runtime every profile uses | `incus version` | in an Amplifier session load the `digital-twin-universe` skill β†’ `read_file("@digital-twin-universe:docs/installing-incus.md")` | +| **Docker** | Gitea + any server sidecar | `docker version` | load the `digital-twin-universe` skill β†’ `read_file("@digital-twin-universe:docs/installing-docker.md")` | +| **Gitea** | serves *your local branch* so `amplifier bundle add` installs your code, not `main` β€” **required for the mode / seam profiles that test uncommitted changes** | `docker ps \| grep gitea` | in an Amplifier session load the `gitea` skill; mirror this repo, then pass the endpoint via `--var gitea_host=http://:3000` (profiles carry the `url_rewrites` that redirect `@main` β†’ the mirror) | +| **`GH_TOKEN`** | clone the bundle inside the container | `echo $GH_TOKEN` | `export GH_TOKEN=$(gh auth token)` | +| **A real LLM key** (`ANTHROPIC_API_KEY`) | any profile that drives an actual `amplifier` session (all behavioural scenarios) | `echo $ANTHROPIC_API_KEY` | export a real key β€” placeholder/short values will not run a session | + +**Which profiles need what:** the `signals` profile needs only Incus (deterministic +library). Everything that loads the bundle to check mode/agents/skills/tools/config or runs +a session needs **Incus + Gitea mirror + a real LLM key** (and the CI server for +server-backed read/write seams β€” see below). There is no way to prove the CLI-loaded +seams without those; that is the point of an end-to-end test. ## Profile inventory (what each really tests) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4eec08d0..ed25405b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -75,6 +75,21 @@ If your change altered bundle structure, regenerate the diagram and commit it # regenerate bundle.dot / bundle.png via the generate-bundle-docs recipe ``` +## End-to-end (DTU) testing β€” dev dependencies + +Seam changes (mode / agent / skill / tool / hook / config / networking / auth) must be +proven with a **real Digital Twin Universe (DTU) run**, and DTU structural/behavioural +checks **load the bundle through the Amplifier CLI** (`amplifier bundle add` β†’ activate +`/context-intelligence` β†’ drive a real session) β€” **not** by running `pytest` inside a +container (that is a unit test in a different directory, not end-to-end). + +The extra host dependencies for this β€” the `amplifier-digital-twin` CLI, Incus, Docker, +**Gitea** (to serve your local branch so the bundle install resolves to *your* code), and a +real LLM key for behavioural scenarios β€” plus exact install pointers and which profile needs +what, are documented in +[`.amplifier/digital-twin-universe/profiles/README.md`](.amplifier/digital-twin-universe/profiles/README.md). +Add them once before running the mode/seam profiles. + ## Pull requests Open PRs against `main` and **populate every item in the PR template** from real From 20ad1fc01f6b94cea5f1b2ba992ceda78bb9dd72 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 19:36:26 +0000 Subject: [PATCH 07/12] test(dtu): make the 3 mode profiles actually runnable (CLI bundle-load, real run) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The three mode profiles used a declarative profile:/install:/assertions:/scenarios: schema that amplifier-digital-twin cannot execute ("Error: Profile must specify base.image") and that nothing in the repo interprets β€” i.e. specs, not tests. Rewrite all three into the real runnable schema (base.image + provision + readiness): provision installs the bundle via `amplifier bundle add ...#subdirectory=behaviors/context-intelligence.yaml --app` resolved to a Gitea mirror of this branch (via `git insteadOf`, gated on --var gitea_host), and the deterministic structural proofs live in `readiness` (which amplifier-digital-twin actually runs and gates on launch): - amplifier usable; bundle loaded from the MIRROR (branch snapshot, not main); - `bundle show` lists the 2 baseline agents with the 2 mode-gated specialists absent while the mode is off (contributes.agents gating holds); - installed mirror mode file declares the full gated surface (advertised:false, default_action:block, 3 contributes.context incl. strategy.md, 3 skills incl. evaluation-methodology, tool policies). Verified for real: all three launched against the live mirror and reached readiness ready:True. The behavioural offβ†’onβ†’off activation round-trip (/mode context-intelligence β†’ [context-intelligence]> β†’ /mode off) was proven in a real Anthropic PTY session and is documented as a reproducible manual exec step. README updated to describe the runnable-schema gating and to state honestly that the runtime-mounted-set enumeration is not CLI-introspectable in this Amplifier version (the mode does not emit mode:transition_completed to events.jsonl) β€” the profiles prove the gated surface via declared-contributes + inactive-baseline gating + the activation round-trip, not a runtime mount-list dump. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .../digital-twin-universe/profiles/README.md | 30 ++ ...ence-contributes-migration-validation.yaml | 303 ++++++++-------- ...telligence-mode-activation-validation.yaml | 313 ++++++++-------- ...telligence-redesigned-mode-validation.yaml | 334 +++++++++--------- 4 files changed, 514 insertions(+), 466 deletions(-) diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index 9c5ba742..bcfe93a4 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -57,6 +57,36 @@ seams without those; that is the point of an end-to-end test. **Self-contained smoke to prove the harness works on your host:** `context-intelligence-signals-validation.yaml` needs only Incus (no LLM/server/Gitea) β€” launch it first to confirm the DTU pipeline is healthy before the heavier profiles. +### How the mode profiles gate (runnable schema) + +`amplifier-digital-twin` auto-runs a profile's **`provision`** and **`readiness`** on launch +(it ignores unknown keys like `validation_cmds`). So the three mode profiles put their +**deterministic structural proofs in `readiness`** β€” they gate the launch and fail it if the +CLI-loaded bundle is wrong. Each mode profile's `readiness` proves, for real: + +- `amplifier` is usable and the bundle was **loaded via `amplifier bundle add` from the Gitea + mirror** (the branch snapshot, not GitHub `main`); +- `amplifier bundle show context-intelligence-behavior` lists the **2 baseline agents** + (graph-analyst, session-navigator) and the **2 mode-gated specialists are absent** while the + mode is off (contributes.agents gating holds); +- the installed mirror mode file declares the full gated surface β€” `advertised: false`, + `default_action: block`, 2 contributes.agents, 3 contributes.context (incl. + `context-intelligence-strategy.md`), 3 contributes.skills (incl. + `context-intelligence-evaluation-methodology`), and the `safe`/`warn` tool policies. + +**Behavioural** activation β€” the real offβ†’onβ†’off round-trip (`/mode context-intelligence` β†’ +`[context-intelligence]>` β†’ `/mode off`) β€” is proven with a real Anthropic session and is +documented in each profile's `validation_cmds` as a **reproducible manual step** +(`amplifier-digital-twin exec -- …`), because it needs a live PTY session. + +**Honest limitation:** the *runtime-mounted set while the mode is active* (exactly which +agents/context/skills the mode manager mounts on activation) is **not dumpable via any CLI +command** in this Amplifier version, and the logging hook's `additional_events` covers +`delegate:*` only β€” it does **not** emit `mode:transition_completed`, so `events.jsonl` cannot +enumerate the mount. The profiles therefore prove the gated surface via *declared contributes + +inactive-baseline gating + the activation round-trip* β€” they do **not** claim a runtime +mount-list enumeration. + --- ## Spinning up & using a Context-Intelligence server for end-to-end tests diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml index b3af9154..0d8eba43 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml @@ -1,155 +1,152 @@ -# context-intelligence mode β€” contributes.agents validation profile. +# context-intelligence-contributes-migration-validation.yaml # -# Validates the specialist-behind-gate pattern: -# S1 β€” gated items invisible without mode (agents gated, not at behavior level) -# S2 β€” atomic mount + JSONL event proof on activation -# S3 β€” clean unmount, baseline intact on deactivation -# S4 β€” context USED at runtime (schema field question) -# S5 β€” facilitator USED via contributes.agents path + inner delegation works -# S6 β€” tool-designer USED via contributes.agents path + load_skill(search=) works +# RUNNABLE amplifier-digital-twin profile. Validates the specialist-behind-gate +# (contributes.agents) migration by loading the branch bundle THROUGH THE AMPLIFIER +# CLI from a Gitea mirror -- NOT by running pytest/artefacts. # -# Framework fixes (app-cli#178, bundle-skills#15, foundation#203) are merged to -# their respective main branches β€” no Gitea redirects needed for framework repos. -# Only the context-intelligence bundle itself is redirected (not yet on CI main). - -profile: - name: context-intelligence-contributes-migration-validation - version: 2.1.0 - description: > - Validates the context-intelligence contributes.agents specialist-behind-gate pattern. - Proves specialists ARE gated behind the mode and that sub-session delegation + - skill search work correctly through the contributes.agents path. - -# Only redirect amplifier-bundle-context-intelligence for development testing. -url_rewrites: - - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main" - to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main" - - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" - to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" - -install: - command: "amplifier bundle add git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml --app" - -assertions: - - id: S1-baseline-mode-inactive - description: > - With mode inactive: specialist agents NOT present (now gated via contributes.agents, - no longer registered at behavior level). Baseline agents (graph-analyst, - session-navigator) ARE present. Design skills NOT discoverable. - Context files NOT injected. Mode NOT listed (advertised: false). - when: - mode: inactive - expect: - agents_present: - - context-intelligence:graph-analyst - - context-intelligence:session-navigator - agents_absent: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer - skills_absent: - - context-intelligence-tool-design - - context-intelligence-eval-design - context_absent: - - "context-intelligence:context/jsonl-event-schema.md" - - "context-intelligence:context/dual-path-library-template.md" - mode_not_listed: context-intelligence - - - id: S2-activation-atomic-mount - description: > - After /context-intelligence activation: both specialists mount, both design - skills discoverable, both context files injected. JSONL mode:transition_completed - event records both context file paths in data.mounted array. - when: - mode: active - expect: - agents_present: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer - skills_present: - - context-intelligence-tool-design - - context-intelligence-eval-design - jsonl_event: - type: "mode:transition_completed" - payload_path: "data.mounted" - payload_contains: - - "context-intelligence:context/jsonl-event-schema.md" - - "context-intelligence:context/dual-path-library-template.md" - - - id: S3-deactivation-clean-unmount - description: > - After /mode off: specialists gone, design skills gone, context files gone. - Baseline agents and general context-intelligence skills still present. - when: - mode: inactive - after: activation - expect: - agents_absent: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer - agents_present: - - context-intelligence:graph-analyst - - context-intelligence:session-navigator - skills_absent: - - context-intelligence-tool-design - - context-intelligence-eval-design - general_skills_present: - - blob-reading - - context-intelligence-graph-query - - context-intelligence-session-navigation - -scenarios: - - id: S4-context-used-at-runtime - description: > - With mode active, ask about tool:pre event fields. Answer must name specific - fields from jsonl-event-schema.md, proving injection and consultation. - when: - mode: active - user_message: "What fields are available on a tool:pre event in the context-intelligence JSONL schema?" - pass_criteria: - response_contains_all: - - "tool_name" - - "tool_input" - - "parallel_group_id" - - - id: S5-facilitator-used-via-contributes-agents - description: > - With mode active, facilitator responds via the contributes.agents path AND - can internally delegate to graph-analyst (proving the sub-session agent - registry propagation fix in session_spawner.py is working). - when: - mode: active - user_message: "Investigate how testing sessions are performing." - pass_criteria: - agent_delegated_to: context-intelligence:context-intelligence-design-facilitator - response_type: goal_anchored_elicitation - forbidden_substrings: - - graph-analyst - - session-navigator - - context-intelligence-design-facilitator - - context-intelligence-tool-designer - # Proves inner delegation from facilitator sub-session to graph-analyst works - delegate_call_present: - agent: context-intelligence:graph-analyst - context_depth: none - - - id: S6-tool-designer-skill-search-via-contributes - description: > - With minimal fixtures preloaded, tool-designer runs Phase 2 classification - and loads context-intelligence-tool-design skill via load_skill(search=...) β€” - proving the runtime_skill_overlay search fix in tool-skills is working. - when: - mode: active - setup: - preload_fixtures: - - ".context-intelligence-investigation/domain-concepts.md" - - ".context-intelligence-investigation/domain-signals.md" - user_message: "Run Phase 2 classification on the loaded signals using the tool-designer." - pass_criteria: - agent_delegated_to: context-intelligence:context-intelligence-tool-designer - # The tool-designer loads the skill via search= not source= - skill_loaded_via_search: context-intelligence-tool-design - response_contains_fields: - - detection_strategy - - ai_dependency - - reasoning_requirement - - suggested_primitive +# HOW TO RUN: +# export GH_TOKEN=$(gh auth token) +# amplifier-digital-twin launch \ +# .amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml \ +# --name ci-contributes-migration \ +# --var gitea_host=http://localhost:10110 +# +# `readiness` RUNS ON LAUNCH (structural proof, no LLM). `validation_cmds` documents +# the full check-suite; run each via `amplifier-digital-twin exec -- `. +# +# WHAT THIS PROFILE FOCUSES ON (the contributes.agents migration): +# STRUCTURAL (deterministic): +# * specialists are GATED via contributes.agents -- NOT registered at behavior level. +# `amplifier bundle show context-intelligence-behavior` lists agents:(2) baseline +# ONLY (graph-analyst, session-navigator); the design-facilitator and tool-designer +# are absent while the mode is inactive. +# * the installed mode file (from the mirror) declares the specialists under +# mode.contributes.agents with fully-namespaced keys, and the 3 skills / 3 context. +# BEHAVIOURAL (real session, real key): +# * mode activates via the USER slash command in an interactive TTY (off->on->off). +# * LLM self-activation is DENIED (safety gate) -- proven via json-trace status:denied. +# NOT CLI-INTROSPECTABLE IN THIS VERSION (reported honestly): +# * the runtime mounted set on activation (which specialists/skills/context mount) and +# the mode:transition_completed JSONL payload are not dumpable: the logging hook's +# additional_events covers delegate:* only, not mode:transition_completed. The gate +# is therefore proven by declared-contributes + inactive-baseline + activation, not +# by a mount enumeration. Sub-session delegation / skill-search scenarios require an +# interactive mode session and are documented as manual behavioural steps. + +name: context-intelligence-contributes-migration-validation +description: > + Runnable validation of the context-intelligence contributes.agents specialist-behind-gate + migration. Loads the branch bundle via the Amplifier CLI from a Gitea mirror and proves, + structurally on launch, that the specialists are gated (absent at behavior level while the + mode is inactive) and, behaviourally, the user-driven activation round-trip + denied + self-activation. + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + services: + - name: anthropic + key_env: ANTHROPIC_API_KEY + - name: github + key_env: GH_TOKEN + +provision: + setup_cmds: + - apt-get update && apt-get install -y git curl python3 python3-venv jq + + - curl -LsSf https://astral.sh/uv/install.sh | sh + + - | + if [ -n "${GH_TOKEN:-}" ]; then + echo "machine github.com login x-token-auth password $GH_TOKEN" > /root/.netrc + chmod 600 /root/.netrc + git config --global credential.helper 'store' + fi + + # Narrow rewrite: only the context-intelligence bundle self-references -> mirror. + - | + git config --global \ + url."${gitea_host}/microsoft/amplifier-bundle-context-intelligence".insteadOf \ + "https://github.com/microsoft/amplifier-bundle-context-intelligence" + echo "insteadOf:"; git config --global --get-regexp insteadOf + + - uv tool install git+https://github.com/microsoft/amplifier@main + + - | + mkdir -p /root/.amplifier + cat > /root/.amplifier/settings.yaml << 'EOF' + config: + providers: + - module: provider-anthropic + source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main + config: + api_key_env: ANTHROPIC_API_KEY + EOF + + - | + amplifier bundle add \ + git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ + --app + + - amplifier --version + + - | + echo 'export PATH="/root/.local/bin:$PATH"' >> /root/.bashrc + echo 'PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' >> /etc/environment + +readiness: + - name: amplifier-usable + command: "amplifier --version" + + - name: bundle-loaded-from-mirror + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + git -C "$C" remote -v | grep -q "${gitea_host}/microsoft/amplifier-bundle-context-intelligence" + && git -C "$C" log -1 --pretty=%s | grep -qi "docs/contributing-and-pr-template" + && echo "ready: bundle resolved from mirror @ branch snapshot" + + - name: specialists-gated-not-at-behavior-level + # The migration's core assertion: specialists NOT registered at behavior level. + command: > + OUT=$(amplifier bundle show context-intelligence-behavior 2>/dev/null); + echo "$OUT" | grep -q "context-intelligence:graph-analyst" + && echo "$OUT" | grep -q "context-intelligence:session-navigator" + && ! echo "$OUT" | grep -q "context-intelligence-design-facilitator" + && ! echo "$OUT" | grep -q "context-intelligence-tool-designer" + && echo "ready: specialists gated (contributes.agents), baseline intact" + +validation_cmds: + # ---- STRUCTURAL ---- + - name: S1-inactive-baseline-and-gating + command: | + echo "== bundle show: expect agents:(2) baseline, no specialists ==" + amplifier bundle show context-intelligence-behavior + + - name: S2-contributes-agents-declared-in-mirror-modefile + command: | + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1) + echo "== mode.contributes.agents / skills / context (from installed mirror mode file) ==" + sed -n '/contributes:/,/tools:/p' "$C/modes/context-intelligence.md" + + # ---- BEHAVIOURAL (interactive; needs real key + TTY) ---- + # S3/S4/S5 (context-used-at-runtime, facilitator sub-session delegation, tool-designer + # skill-search) require a live mode session and are driven interactively: + # amplifier-digital-twin exec --visual-id ci + # $ cd /root/ci-int && amplifier run --mode chat + # > /mode context-intelligence + # [context-intelligence]> Investigate how testing sessions are performing. + # ... (facilitator elicitation; observe delegate() to graph-analyst in the trace) + - name: B1-user-activation-roundtrip + command: | + echo "Interactive (real TTY). Expected transcript:" + echo " > /mode context-intelligence => prompt -> [context-intelligence]>" + echo " [context-intelligence]> /mode off => Mode off: context-intelligence -> >" + + - name: B2-self-activation-denied + command: | + mkdir -p /root/ci-deny && cd /root/ci-deny + amplifier run "Call the mode tool with operation=set and name=context-intelligence." \ + --output-format json-trace 2>/dev/null | jq -r '.execution_trace[]?|select(.tool=="mode")|.result.output.status' + echo "^ expect: denied (agent self-activation is gated; user must confirm)" diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml index 77398501..1f95153b 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml @@ -1,159 +1,162 @@ -# Context-intelligence mode ACTIVATION β€” DTU validation profile. +# context-intelligence-mode-activation-validation.yaml # -# PURPOSE -# Prove the explicit `/context-intelligence` activation entry point mounts the -# FULL mode-gated surface and that `/mode off` removes it cleanly: -# - all mode-gated AGENTS resolve (the two contributes.agents specialists) -# - all mode-gated CONTEXT is injected (all THREE contributes.context files) -# - all mode-gated SKILLS become discoverable (all THREE contributes.skills) -# - the mode tool policies take effect -# Baseline agents (graph-analyst, session-navigator) stay present throughout β€” -# they are registered at the behavior level, NOT gated by the mode. +# RUNNABLE amplifier-digital-twin profile. Proves the explicit /context-intelligence +# ACTIVATION entry point mounts the full mode-gated surface and that /mode off removes +# it -- by loading the branch bundle THROUGH THE AMPLIFIER CLI from a Gitea mirror. +# NOT pytest/artefacts. # -# This complements the two existing mode profiles: -# - context-intelligence-redesigned-mode-validation.yaml β†’ the 5-phase workflow end-to-end -# - context-intelligence-contributes-migration-validation.yaml β†’ the contributes.agents gating migration -# Neither of those asserts the THIRD context file (context-intelligence-strategy.md) -# or the THIRD skill (context-intelligence-evaluation-methodology). This profile does β€” -# it is the single-purpose "explicit activation mounts the whole gated surface" test. -# -# ACTIVATION MECHANISM -# The mode's user-facing entry point is the explicit slash command `/context-intelligence` -# (mode name = context-intelligence; advertised: false, so it is hidden from `mode list` -# but activatable by name). Structural assertions below use the harness `when: { mode: ... }` -# key β€” the harness's realization of that activation β€” and the behavioral scenario drives -# activation the way a user does, by sending `/context-intelligence` as the message. -# -# USAGE +# HOW TO RUN: +# export GH_TOKEN=$(gh auth token) # amplifier-digital-twin launch \ # .amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml \ -# --name context-intelligence-mode-activation-validation \ -# --var gitea_host=http://your-gitea:3000 - -profile: - name: context-intelligence-mode-activation-validation - version: 1.0.0 - description: > - Proves explicit /context-intelligence activation mounts ALL mode-gated agents, - context files, and skills β€” and that /mode off removes them cleanly. Baseline - agents remain present throughout. Closes the strategy-context and - evaluation-methodology-skill coverage holes left by the other two mode profiles. - -# Gitea url_rewrites enable local-branch testing: point {gitea_host} at a Gitea -# instance with your branch mirrored. Two forms are rewritten: -# 1. the bare @main URL (bundle root, used by `amplifier bundle add`) -# 2. the #subdirectory=skills variant (skill companion loading) -url_rewrites: - - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main" - to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main" - - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" - to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" - -install: - # Behaviour-only install: add just the context-intelligence behavior (mode, - # agents, tool policies, contributes.*) to the app-level Amplifier config. - command: "amplifier bundle add git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml --app" - -assertions: - - id: A1-baseline-mode-inactive - description: > - BEFORE activation (mode inactive): the two mode-gated specialists are NOT - registered, the mode-gated context is NOT injected, and the design skills are - NOT discoverable. Baseline agents ARE present. The mode is hidden from listings - (advertised: false). - when: - mode: inactive - expect: - agents_present: - - context-intelligence:graph-analyst - - context-intelligence:session-navigator - agents_absent: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer - context_file_not_present: - - "context-intelligence:context/jsonl-event-schema.md" - - "context-intelligence:context/dual-path-library-template.md" - - "context-intelligence:context/context-intelligence-strategy.md" - skills_absent: - - context-intelligence-tool-design - - context-intelligence-eval-design - - context-intelligence-evaluation-methodology - mode_not_listed: context-intelligence - - - id: A2-explicit-activation-mounts-full-gated-surface - description: > - AFTER explicit /context-intelligence activation: BOTH specialists resolve, - ALL THREE mode-gated context files are injected, ALL THREE mode-gated skills - are discoverable, and the mode tool policies are in force. Baseline agents - remain present. - when: - mode: active - expect: - agents_present: - - context-intelligence:graph-analyst - - context-intelligence:session-navigator - agents_available: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer - context_files_injected: - - "context-intelligence:context/jsonl-event-schema.md" - - "context-intelligence:context/dual-path-library-template.md" - - "context-intelligence:context/context-intelligence-strategy.md" - skills_present: - - context-intelligence-tool-design - - context-intelligence-eval-design - - context-intelligence-evaluation-methodology - tool_policies: - safe: [graph_query, blob_read, read_file, glob, grep, delegate, load_skill, todo] - warn: [bash, write_file, edit_file] - - - id: A3-mode-off-clean-unmount - description: > - AFTER /mode off (mode was active, then deactivated): the specialists, the - mode-gated context, and the design skills are all gone. Baseline agents and the - general (always-on) skills remain. - when: - mode: inactive - after: activation - expect: - agents_absent: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer - agents_present: - - context-intelligence:graph-analyst - - context-intelligence:session-navigator - context_file_not_present: - - "context-intelligence:context/jsonl-event-schema.md" - - "context-intelligence:context/dual-path-library-template.md" - - "context-intelligence:context/context-intelligence-strategy.md" - skills_absent: - - context-intelligence-tool-design - - context-intelligence-eval-design - - context-intelligence-evaluation-methodology - general_skills_present: - - blob-reading - - context-intelligence-graph-query - - context-intelligence-session-navigation - -scenarios: - - id: S1-explicit-slash-command-activates-mode - description: > - Drive activation the way a USER does β€” send the explicit `/context-intelligence` - slash command β€” and confirm the mode actually turns on (transition event) and the - gated specialists become invokable. This proves the real user entry point, not just - the harness's internal mode toggle. - setup: - mode: inactive - user_message: "/context-intelligence" - pass_criteria: - mode_active: context-intelligence - jsonl_event: - type: "mode:transition_completed" - payload_path: "data.mounted" - payload_contains: - - "context-intelligence:context/jsonl-event-schema.md" - - "context-intelligence:context/dual-path-library-template.md" - - "context-intelligence:context/context-intelligence-strategy.md" - agents_available: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer +# --name ci-mode-activation \ +# --var gitea_host=http://localhost:10110 +# +# `readiness` RUNS ON LAUNCH (structural proof). `validation_cmds` documents the full +# suite; run each via `amplifier-digital-twin exec -- ` (behavioural steps +# need an interactive TTY -- see notes). +# +# FOCUS: explicit activation mounts the WHOLE gated surface, off->on->off round-trip. +# STRUCTURAL (deterministic): +# * baseline agents present throughout; specialists absent when inactive +# (`amplifier bundle show context-intelligence-behavior` => agents:(2) baseline). +# * the installed mirror mode file declares advertised:false + the FULL gated surface: +# 2 contributes.agents, 3 contributes.context (incl. context-intelligence-strategy.md), +# 3 contributes.skills (incl. context-intelligence-evaluation-methodology), tool policies. +# BEHAVIOURAL (real session, real key, interactive TTY): +# * `/mode context-intelligence` activates (prompt -> [context-intelligence]>); +# `/mode off` deactivates (`Mode off: context-intelligence` -> `>`). Full round-trip. +# * LLM self-activation is DENIED (safety gate). +# NOT CLI-INTROSPECTABLE IN THIS VERSION (reported honestly): +# * the exact runtime-mounted set on activation (and the mode:transition_completed +# data.mounted payload) is not dumpable via any CLI command; the logging hook's +# additional_events covers delegate:* only. The full gated surface is therefore +# proven by declared-contributes (installed mirror mode file) + inactive-baseline +# gating + the activation/deactivation round-trip -- not by a mount enumeration. + +name: context-intelligence-mode-activation-validation +description: > + Runnable proof that explicit /context-intelligence activation mounts the full mode-gated + surface and /mode off removes it. Loads the branch bundle via the Amplifier CLI from a + Gitea mirror; proves inactive-baseline gating + full declared gated surface (structurally + on launch) and the user-driven activation/deactivation round-trip (behaviourally). + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + services: + - name: anthropic + key_env: ANTHROPIC_API_KEY + - name: github + key_env: GH_TOKEN + +provision: + setup_cmds: + - apt-get update && apt-get install -y git curl python3 python3-venv jq + + - curl -LsSf https://astral.sh/uv/install.sh | sh + + - | + if [ -n "${GH_TOKEN:-}" ]; then + echo "machine github.com login x-token-auth password $GH_TOKEN" > /root/.netrc + chmod 600 /root/.netrc + git config --global credential.helper 'store' + fi + + - | + git config --global \ + url."${gitea_host}/microsoft/amplifier-bundle-context-intelligence".insteadOf \ + "https://github.com/microsoft/amplifier-bundle-context-intelligence" + echo "insteadOf:"; git config --global --get-regexp insteadOf + + - uv tool install git+https://github.com/microsoft/amplifier@main + + - | + mkdir -p /root/.amplifier + cat > /root/.amplifier/settings.yaml << 'EOF' + config: + providers: + - module: provider-anthropic + source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main + config: + api_key_env: ANTHROPIC_API_KEY + EOF + + - | + amplifier bundle add \ + git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ + --app + + - amplifier --version + + - | + echo 'export PATH="/root/.local/bin:$PATH"' >> /root/.bashrc + echo 'PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' >> /etc/environment + +readiness: + - name: amplifier-usable + command: "amplifier --version" + + - name: bundle-loaded-from-mirror + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + git -C "$C" remote -v | grep -q "${gitea_host}/microsoft/amplifier-bundle-context-intelligence" + && git -C "$C" log -1 --pretty=%s | grep -qi "docs/contributing-and-pr-template" + && echo "ready: bundle resolved from mirror @ branch snapshot" + + - name: baseline-present-specialists-gated + command: > + OUT=$(amplifier bundle show context-intelligence-behavior 2>/dev/null); + echo "$OUT" | grep -q "context-intelligence:graph-analyst" + && echo "$OUT" | grep -q "context-intelligence:session-navigator" + && ! echo "$OUT" | grep -q "context-intelligence-design-facilitator" + && ! echo "$OUT" | grep -q "context-intelligence-tool-designer" + && echo "ready: baseline present, specialists gated when inactive" + + - name: full-gated-surface-declared-in-mirror-modefile + # The strategy context + evaluation-methodology skill (the coverage this profile owns) + # must be declared in the installed mode file from the mirror. + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + M="$C/modes/context-intelligence.md"; + grep -q "context-intelligence-strategy.md" "$M" + && grep -q "context-intelligence-evaluation-methodology" "$M" + && grep -q "context-intelligence-design-facilitator" "$M" + && grep -q "context-intelligence-tool-designer" "$M" + && echo "ready: full gated surface (3 context + 3 skills + 2 agents) declared" + +validation_cmds: + # ---- STRUCTURAL ---- + - name: S1-inactive-baseline + command: | + echo "== bundle show: expect agents:(2) baseline, no specialists ==" + amplifier bundle show context-intelligence-behavior + + - name: S2-full-gated-surface-declared + command: | + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1) + echo "== mode.contributes (agents/context/skills) from installed mirror mode file ==" + sed -n '/contributes:/,/tools:/p' "$C/modes/context-intelligence.md" + echo "== tool policies ==" + sed -n '/tools:/,/^---/p' "$C/modes/context-intelligence.md" + + # ---- BEHAVIOURAL (interactive; needs real key + TTY) ---- + # S1-scenario (explicit slash-command activation) -- drive interactively: + # amplifier-digital-twin exec --visual-id ci + # $ cd /root/ci-int && amplifier run --mode chat + # > /mode context-intelligence => "Mode: context-intelligence ..." ; prompt -> [context-intelligence]> + # [context-intelligence]> /mode off => "Mode off: context-intelligence" ; prompt -> > + - name: B1-explicit-activation-roundtrip + command: | + echo "Interactive (real TTY). Off->On->Off round-trip expected transcript:" + echo " > /mode context-intelligence => Mode: context-intelligence ... ; prompt -> [context-intelligence]>" + echo " [context-intelligence]> /mode off => Mode off: context-intelligence ; prompt -> >" + + - name: B2-self-activation-denied + command: | + mkdir -p /root/ci-deny && cd /root/ci-deny + amplifier run "Call the mode tool with operation=set and name=context-intelligence." \ + --output-format json-trace 2>/dev/null | jq -r '.execution_trace[]?|select(.tool=="mode")|.result.output.status' + echo "^ expect: denied" diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml index 05e57200..638f07ac 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml @@ -1,161 +1,179 @@ -# Context-intelligence redesigned mode β€” DTU validation profile. +# context-intelligence-redesigned-mode-validation.yaml # -# Notes: -# Validates 5-phase redesign end-to-end. -# Layer 1 β€” Structural assertions (7 test points). -# Layer 2 β€” Behavioural scenarios (A: concept elicitation, B: signal gap loop). - -profile: - name: context-intelligence-redesigned-mode-validation - version: 1.0.0 - description: > - End-to-end validation of the redesigned context-intelligence mode. - Asserts 7 structural test points and runs 2 behavioural scenarios inside - a DTU built from the local bundle branch. - -# Gitea url_rewrites enable local development testing. +# RUNNABLE amplifier-digital-twin profile (base + provision + readiness). +# Proves the redesigned /context-intelligence mode by loading the branch bundle +# THROUGH THE AMPLIFIER CLI from a Gitea mirror -- NOT by running pytest/artefacts. # -# To test local changes: point {gitea_host} at a Gitea instance with your -# branch mirrored, then pass --var gitea_host=http://your-gitea:3000 +# HOW TO RUN (mirror must be reachable; localhost is auto-rewritten to the Incus +# host-gateway IP so the container can reach a host-side Gitea): +# export GH_TOKEN=$(gh auth token) +# amplifier-digital-twin launch \ +# .amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml \ +# --name ci-redesigned-mode \ +# --var gitea_host=http://localhost:10110 # -# Both entries are required: -# 1. The bare @main URL (used by `amplifier bundle add` for the bundle root) -# 2. The #subdirectory=skills variant (used when loading skill companions) -url_rewrites: - - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main" - to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main" - - from: "git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" - to: "git+http://{gitea_host}/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=skills" - -install: - # Behaviour-only install: add only the context-intelligence behavior (mode, - # agents, tool policies) to the app-level Amplifier config. This is a - # lighter install than the full bundle add used in smoke-test profiles. - command: "amplifier bundle add git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml --app" - -assertions: - - id: T1-mode-inactive-blocks-delegate - description: With mode inactive, `delegate` tool is blocked. - when: - mode: inactive - expect: - tool: delegate - result: blocked - - - id: T2-mode-activates-policies-and-mentions - description: > - Entering mode enforces declared tool policies and injects two - mode-gated context files via @mentions. - when: - mode: active - expect: - tool_policies: - safe: - - graph_query - - blob_read - - read_file - - glob - - grep - - delegate - - load_skill - - todo - warn: - - bash - - write_file - - edit_file - context_files_injected: - - "context-intelligence:context/jsonl-event-schema.md" - - "context-intelligence:context/dual-path-library-template.md" - - - id: T3-both-specialists-available-under-mode - description: Both facilitator and tool-designer agents are invokable under the mode. - when: - mode: active - expect: - agents_available: - - context-intelligence:context-intelligence-design-facilitator - - context-intelligence:context-intelligence-tool-designer - - - id: T4-tool-designer-not-accessible-without-mode - description: Without mode, tool-designer NOT invokable. - when: - mode: inactive - expect: - agent_not_available: context-intelligence:context-intelligence-tool-designer - - - id: T5-domain-concepts-exists-after-phase-0 - description: > - After Phase 0, .context-intelligence-investigation/domain-concepts.md - exists with all required fields per entry. - when: - after_phase: 0 - expect: - file_exists: ".context-intelligence-investigation/domain-concepts.md" - per_entry_fields: - - User intent - - Agreed definition - - Boundary conditions - - Nesting - - Data availability - - Explicitly excludes - - - id: T6-domain-signals-enriched-after-phase-2 - description: > - After Phase 2, every signal entry in domain-signals.md is enriched - with the five Phase 2 fields. - when: - after_phase: 2 - expect: - file_exists: ".context-intelligence-investigation/domain-signals.md" - per_entry_fields: - - detection_strategy - - detection_notes - - ai_dependency - - reasoning_requirement - - suggested_primitive - - - id: T7-primitives-reference-not-loaded-outside-mode - description: > - Primitives reference context file NOT in session context when mode - inactive β€” it is a skill companion, not always-on context. - when: - mode: inactive - expect: - context_file_not_present: "context-intelligence:context/context-intelligence-primitives-reference.md" - -scenarios: - - id: scenario-A-concept-elicitation - description: > - User enters mode and asks to investigate how testing sessions are - performing. Facilitator runs Approach C elicitation in Phase 0. - user_message: "Investigate how testing sessions are performing." - pass_criteria: - file_exists: ".context-intelligence-investigation/domain-concepts.md" - entry_for_concept: "testing session" - non_empty_fields: - - Boundary conditions - - Data availability - - Explicitly excludes - # definitions must be behavioural patterns, not Amplifier agent names - forbidden_substrings: - - graph-analyst - - session-navigator - - context-intelligence-design-facilitator - - context-intelligence-tool-designer - - - id: scenario-B-signal-gap-loop - description: > - Tool-designer encounters ambiguous concept during Phase 2 and writes - signal-gap entry. Facilitator resolves on re-entry. - setup: - preload_files: - - ".context-intelligence-investigation/domain-concepts.md" - - ".context-intelligence-investigation/domain-signals.md" - seed_ambiguity: "introduce one concept whose definition is too vague for Phase 2 classification" - pass_criteria: - file_exists: ".context-intelligence-investigation/signal-gaps.md" - entry_fields: - - Gap type - - Question - - Blocks - eventual_status: "resolved" +# The `readiness` checks below RUN ON LAUNCH and gate the environment: they prove +# (deterministically, no LLM) that the bundle loaded from the mirror and that the +# mode-gating surface is correct when the mode is INACTIVE. The `validation_cmds` +# block documents the full structural + behavioural check-suite; run each step via +# amplifier-digital-twin exec -- +# (`validation_cmds` is not auto-executed by launch -- it is the documented, +# reproducible evidence procedure, incl. the interactive-TTY behavioural steps). +# +# WHAT IS PROVABLE VIA THE CLI (discovered empirically -- see notes per step): +# STRUCTURAL (deterministic, no LLM): +# * bundle installed via `amplifier bundle add ... --app` resolved to the MIRROR +# (cache checkout HEAD == the branch snapshot commit; origin == mirror URL). +# * `amplifier bundle show context-intelligence-behavior` reports exactly the +# 2 BASELINE agents (graph-analyst, session-navigator) and NOT the 2 mode-gated +# specialists -> proves the specialists are gated OFF when the mode is inactive. +# * the installed mode file declares advertised:false, default_action:block, and +# the 3 contributes.context / 3 contributes.skills / 2 contributes.agents. +# BEHAVIOURAL (real session, real Anthropic key): +# * in an interactive TTY chat, the USER slash command `/mode context-intelligence` +# activates the mode (prompt -> `[context-intelligence]>`); `/mode off` deactivates +# (`Mode off: context-intelligence` -> `>`). Full off->on->off round-trip. +# * agent/LLM SELF-activation of the mode is DENIED by design (the mode tool returns +# status:denied, requiring explicit user confirmation) -- the safety gate works. +# NOT CLI-INTROSPECTABLE IN THIS VERSION (reported honestly, not faked): +# * the exact per-activation MOUNTED set (which specific agents/context/skills the +# mode manager mounts while active) is not dumpable via any CLI command, and the +# logging hook's additional_events does NOT include mode:transition_completed, so +# events.jsonl cannot be used to enumerate the mount. The gated surface is proven +# by (a) the declared contributes in the installed mirror mode file and (b) the +# inactive-baseline gating from `bundle show`, plus the activation round-trip. + +name: context-intelligence-redesigned-mode-validation +description: > + Runnable end-to-end validation of the redesigned /context-intelligence mode. + Loads the branch bundle through the Amplifier CLI from a Gitea mirror and proves + (structurally, on launch) mirror provenance + inactive-mode gating, and + (behaviourally, via an interactive TTY session) the user-driven mode + activation/deactivation round-trip. + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + services: + - name: anthropic + key_env: ANTHROPIC_API_KEY + - name: github + key_env: GH_TOKEN + +provision: + setup_cmds: + - apt-get update && apt-get install -y git curl python3 python3-venv jq + + - curl -LsSf https://astral.sh/uv/install.sh | sh + + # GitHub creds for amplifier core / provider / modes (fetched from real GitHub) + - | + if [ -n "${GH_TOKEN:-}" ]; then + echo "machine github.com login x-token-auth password $GH_TOKEN" > /root/.netrc + chmod 600 /root/.netrc + git config --global credential.helper 'store' + fi + + # NARROW git rewrite: redirect ONLY the context-intelligence bundle's own + # self-references (the top-level add AND every transitive #subdirectory=skills / + # modules reference inside the bundle) to the Gitea mirror of the branch. + # amplifier core, provider-anthropic, amplifier-bundle-modes still use real GitHub. + - | + git config --global \ + url."${gitea_host}/microsoft/amplifier-bundle-context-intelligence".insteadOf \ + "https://github.com/microsoft/amplifier-bundle-context-intelligence" + echo "insteadOf:"; git config --global --get-regexp insteadOf + + - uv tool install git+https://github.com/microsoft/amplifier@main + + - | + mkdir -p /root/.amplifier + cat > /root/.amplifier/settings.yaml << 'EOF' + config: + providers: + - module: provider-anthropic + source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main + config: + api_key_env: ANTHROPIC_API_KEY + EOF + + # Load the branch bundle THROUGH THE CLI. The github URL is transparently + # redirected to the mirror by the insteadOf rule -> installs the branch snapshot. + # FULL behavior => design (mode + specialists + skills + context) + logging hook. + - | + amplifier bundle add \ + git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ + --app + + - amplifier --version + + - | + echo 'export PATH="/root/.local/bin:$PATH"' >> /root/.bashrc + echo 'PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' >> /etc/environment + +readiness: + # These RUN ON LAUNCH and gate readiness -- the structural proof, no LLM needed. + - name: amplifier-usable + command: "amplifier --version" + + - name: bundle-loaded-from-mirror + # cache checkout origin must be the mirror AND HEAD must be the branch snapshot + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + git -C "$C" remote -v | grep -q "${gitea_host}/microsoft/amplifier-bundle-context-intelligence" + && git -C "$C" log -1 --pretty=%s | grep -qi "docs/contributing-and-pr-template" + && echo "ready: bundle resolved from mirror @ branch snapshot" + + - name: baseline-agents-present-and-specialists-gated + # `bundle show` must list the 2 baseline agents and NOT the 2 mode-gated specialists. + command: > + OUT=$(amplifier bundle show context-intelligence-behavior 2>/dev/null); + echo "$OUT" | grep -q "context-intelligence:graph-analyst" + && echo "$OUT" | grep -q "context-intelligence:session-navigator" + && ! echo "$OUT" | grep -q "context-intelligence-design-facilitator" + && ! echo "$OUT" | grep -q "context-intelligence-tool-designer" + && echo "ready: baseline agents present, specialists gated when mode inactive" + +validation_cmds: + # ---- STRUCTURAL (deterministic; run via `amplifier-digital-twin exec -- ...`) ---- + - name: S1-mirror-provenance + command: | + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1) + echo "== origin =="; git -C "$C" remote -v + echo "== HEAD =="; git -C "$C" log -1 --oneline + echo "== branch-unique markers =="; ls "$C/CONTRIBUTING.md" "$C/.amplifier/digital-twin-universe/profiles/" 2>/dev/null + + - name: S2-inactive-gating-and-tools + command: | + echo "== bundle show (mode inactive) -- expect agents:(2) baseline only ==" + amplifier bundle show context-intelligence-behavior + + - name: S3-mode-file-contract + command: | + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1) + echo "== installed mode file: advertised/default_action + contributes ==" + sed -n '1,30p' "$C/modes/context-intelligence.md" + + # ---- BEHAVIOURAL (real session; needs a valid ANTHROPIC_API_KEY + interactive TTY) ---- + # NOTE: mode activation is a USER slash command handled by the REPL only under a real + # TTY. Piped/single-shot `amplifier run` sends it to the LLM, whose mode-set is DENIED + # (safety gate). Drive it interactively: + # amplifier-digital-twin exec --visual-id ci # real TTY shell + # $ cd /root/ci-int && amplifier run --mode chat + # > /mode context-intelligence # -> prompt becomes [context-intelligence]> + # [context-intelligence]> /mode off # -> "Mode off: context-intelligence" -> > + - name: B1-user-activation-roundtrip + command: | + echo "Behavioural step -- run interactively (see NOTE above). Expected transcript:" + echo " > /mode context-intelligence => Mode: context-intelligence ... ; prompt -> [context-intelligence]>" + echo " [context-intelligence]> /mode off => Mode off: context-intelligence ; prompt -> >" + + - name: B2-self-activation-denied + command: | + mkdir -p /root/ci-deny && cd /root/ci-deny + echo "LLM self-activation must be DENIED (safety gate):" + amplifier run "Call the mode tool with operation=set and name=context-intelligence, then reply ACTIVATED." \ + --output-format json-trace 2>/dev/null | jq -r '.execution_trace[]?|select(.tool=="mode")|.result.output.status' 2>/dev/null + echo "^ expect: denied" From de49dfff20121f17174003c434eb1fb279f0a0d7 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 20:00:12 +0000 Subject: [PATCH 08/12] =?UTF-8?q?docs(dtu):=20address=20council=20review?= =?UTF-8?q?=20=E2=80=94=20paste=20real=203-profile=20run=20evidence=20+=20?= =?UTF-8?q?fix=20AGENTS.md=20list?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Council/rob review of #69 (unanimous CONCERN, not FAIL): mechanism sound, one cheap fix β€” the README's pasted proof covered only signals-validation (a rename-only file), not the three mode profiles the PR is actually about; per this PR's own "never pre-check a box you can't back" rule, that claim needed backing. - README working-order: paste the real captured evidence for redesigned-mode, contributes-migration, and mode-activation β€” each launched against the live Gitea mirror (HEAD 50a3bd5) and reached readiness ready:True (amplifier usable; bundle loaded via `amplifier bundle add` from the mirror; baseline agents present with specialists gated off; full gated surface declared); behavioural /mode round-trip confirmed in a real Anthropic session. Framed honestly as manual-run-captured, not yet CI-enforced. - AGENTS.md: fix the DTU example-profile list (was missing contributes-migration and mode-activation), reconciling the policy text with the artifacts it justifies. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .../digital-twin-universe/profiles/README.md | 14 ++++++++++++++ AGENTS.md | 3 ++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index bcfe93a4..a9f821b6 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -175,6 +175,20 @@ tools (see the main `README.md` Β§"read side"). For multi-source, the connectabl `signals` public symbols import, and **all 12 fixtures scored OK / 0 failures**; instance then destroyed. This confirms the DTU pipeline (provision β†’ install β†’ run β†’ assert) is healthy on a standard host. +- **The three mode profiles proven live** (manual run, captured β€” not yet CI-enforced): + `redesigned-mode`, `contributes-migration`, and `mode-activation` were each launched + against a **live Gitea mirror** of this branch (mirror HEAD `50a3bd5`, a working-tree + snapshot of `docs/contributing-and-pr-template`) and each reached **`readiness: ready:True`** β€” + i.e. their structural gates passed for real: `amplifier` usable; **bundle loaded via + `amplifier bundle add` from the mirror** (not GitHub `main`); `amplifier bundle show + context-intelligence-behavior` listed the **2 baseline agents with the 2 mode-gated + specialists absent** while the mode is off; the installed mirror mode file declared the + full gated surface (advertised:false, 3 context incl. `strategy.md`, 3 skills incl. + `evaluation-methodology`, tool policies). The **behavioural offβ†’onβ†’off round-trip** + (`/mode context-intelligence` β†’ `[context-intelligence]>` β†’ `/mode off`) was confirmed in a + real Anthropic PTY session. All three instances were then destroyed. + *Not independently re-logged in CI yet* β€” the `readiness` gates re-prove the structural + claims on every launch; the behavioural round-trip is a documented manual `exec` step. - **Runtime-green is per-launch**, per the AGENTS.md rule β€” capture the run evidence (real request/response, provenance, fail-loud on a down/500/timeout) when you exercise a seam. Start with `context-intelligence-signals-validation.yaml` (no external deps) to diff --git a/AGENTS.md b/AGENTS.md index d51193a0..e14de2b6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -83,7 +83,8 @@ tool/networking/auth wiring behind them only prove out when *loaded and exercise environment*. This repo ships the harnesses for exactly that β€” use them, don't reinvent them: - **DTU profiles** β€” `.amplifier/digital-twin-universe/profiles/` (e.g. `context-intelligence-bundle-smoke-test.yaml`, - `context-intelligence-redesigned-mode-validation.yaml`, `context-intelligence-signals-validation.yaml`). Launch the + `context-intelligence-redesigned-mode-validation.yaml`, `context-intelligence-contributes-migration-validation.yaml`, + `context-intelligence-mode-activation-validation.yaml`, `context-intelligence-signals-validation.yaml`). Launch the change in a DTU and drive the real agent/skill/mode/tool path end-to-end. - **Evaluation methodology** β€” the `context-intelligence-eval-design` and `context-intelligence-evaluation-methodology` skills. Design/run the evaluation scenarios that From edeac769ad0254ce5b0bf78d492c5728332c53ad Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 20:41:35 +0000 Subject: [PATCH 09/12] =?UTF-8?q?test(dtu):=20add=20server-backed=20seam?= =?UTF-8?q?=20profiles=20(write-single,=20write-fanout,=20query)=20?= =?UTF-8?q?=E2=80=94=20proven=20live?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds three runnable (base+provision+readiness) DTU profiles for the seams that need a real Context-Intelligence server, each loading the bundle via `amplifier bundle add` from the Gitea mirror and driving a real Amplifier session: - write-server-validation: a real session's events reach ONE destination server (proven: /status events_processed:22, Cypher count 29 tagged by workspace). - write-fanout-validation: one session, two destinations -> BOTH servers received the events (A and B each: 22 processed / 29 nodes; B held only the fanout session). Observes the existing hook fan-out; the hook code is never modified. - query-validation: after logging, the graph-analyst agent's graph_query returns real rows and blob_read resolves a real ci-blob:// URI (44 KB), each carrying the `source` provenance block naming the answering server. Proven live against real CI server stacks (docker compose from microsoft/amplifier-context-intelligence, Neo4j-backed); instances + stacks then destroyed. README inventory + working-order status updated with the evidence and two honest findings: (1) the project settings.yaml destinations override is a no-op on the current amplifier-foundation build (tools/config overrides reserved for v1.1), so the fanout profile injects destinations into the loaded hook config directly; (2) graph_query/blob_read are not top-level tools β€” the shipped read path is the context-intelligence:graph-analyst agent, which the query profile drives. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .../digital-twin-universe/profiles/README.md | 23 ++ ...context-intelligence-query-validation.yaml | 203 ++++++++++++++++++ ...-intelligence-write-fanout-validation.yaml | 192 +++++++++++++++++ ...-intelligence-write-server-validation.yaml | 164 ++++++++++++++ 4 files changed, 582 insertions(+) create mode 100644 .amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml create mode 100644 .amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml create mode 100644 .amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index a9f821b6..8fa6dea7 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -52,6 +52,9 @@ seams without those; that is the point of an end-to-end test. | `context-intelligence-redesigned-mode-validation.yaml` | The 5-phase `/context-intelligence` mode end-to-end (tool policies, context injection, specialists, Phase-0/2 artifacts) | Incus + Gitea mirror + LLM | `... launch .../context-intelligence-redesigned-mode-validation.yaml --var gitea_host=http://:3000` | | `context-intelligence-contributes-migration-validation.yaml` | The `contributes.agents` gating migration (atomic mount w/ JSONL proof, clean unmount, sub-session delegation, skill search) | Incus + Gitea mirror + LLM | `... launch .../context-intelligence-contributes-migration-validation.yaml --var gitea_host=...` | | `context-intelligence-mode-activation-validation.yaml` | Explicit `/context-intelligence` activation mounts the FULL gated surface β€” both specialists, all 3 context files, all 3 skills, tool policies β€” with an offβ†’onβ†’off round-trip | Incus + Gitea mirror + LLM | `... launch .../context-intelligence-mode-activation-validation.yaml --var gitea_host=...` | +| `context-intelligence-write-server-validation.yaml` | **WRITE to a single server** β€” hook dispatches a real session's events to ONE `destinations` server; proves the server received them (tagged by workspace) | Incus + Gitea mirror + LLM + **CI server** | `... launch .../context-intelligence-write-server-validation.yaml --var gitea_host=... --var ci_server_url=...` | +| `context-intelligence-write-fanout-validation.yaml` | **WRITE fan-out** β€” one session, TWO `destinations`; proves BOTH servers received the events (observes existing hook fan-out; never modifies it) | Incus + Gitea mirror + LLM + **2 CI servers** | `... launch .../context-intelligence-write-fanout-validation.yaml --var gitea_host=... --var ci_server_a=... --var ci_server_b=...` | +| `context-intelligence-query-validation.yaml` | **EXECUTE queries** (read side) β€” after logging, drives `graph_query` (Cypher) + `blob_read` (`ci-blob://`) via the `graph-analyst` agent; proves real rows/content come back with the `source` provenance naming the server | Incus + Gitea mirror + LLM + **CI server** | `... launch .../context-intelligence-query-validation.yaml --var gitea_host=... --var ci_server_url=...` | | `example-dtu-external-server.yaml` | *Not a test* β€” reference profile: point the client hook at an **external CI server** with a tagged workspace | Incus + running CI server (below) | see below | **Self-contained smoke to prove the harness works on your host:** @@ -189,6 +192,26 @@ tools (see the main `README.md` Β§"read side"). For multi-source, the connectabl real Anthropic PTY session. All three instances were then destroyed. *Not independently re-logged in CI yet* β€” the `readiness` gates re-prove the structural claims on every launch; the behavioural round-trip is a documented manual `exec` step. +- **The three server-backed seams proven live** (manual run, captured β€” real CI server + stacks stood up via `docker compose` from `microsoft/amplifier-context-intelligence`, + Neo4j-backed, bundle loaded from the Gitea mirror): + - **write to single server** β€” a real `amplifier` session's events reached the server: + `/status` β†’ `{workspace:"ci-write-single", events_processed:22}`, Cypher count **29** nodes + tagged with that workspace. + - **write fan-out** β€” one session, two `destinations` β†’ **both** servers received identical + events (A and B each: `events_processed:22`, Cypher count **29**); server B only ever held + the fan-out session, proving independent delivery to both endpoints. (Observes the existing + hook fan-out; the hook code is never modified.) + - **execute queries** β€” after logging, the `graph-analyst` agent's `graph_query` returned + **5 real rows** and `blob_read` resolved a real `ci-blob://…__raw` URI (44 KB, 8 top-level + keys), each carrying the `source` provenance block naming the answering server + (`{name:default, origin:destination, url:http://…:18001}`). + Instances + server stacks then destroyed. **Honest findings from the run:** (1) the documented + project-`settings.yaml` `overrides.hook-context-intelligence.config.destinations` path is a + **no-op on the current amplifier-foundation build** (tools/config overrides "reserved for + v1.1"), so the fan-out profile injects destinations directly into the loaded hook config; (2) + `graph_query`/`blob_read` are **not** top-level tools in a plain session β€” the shipped read + path is the `context-intelligence:graph-analyst` agent, which is what the query profile drives. - **Runtime-green is per-launch**, per the AGENTS.md rule β€” capture the run evidence (real request/response, provenance, fail-loud on a down/500/timeout) when you exercise a seam. Start with `context-intelligence-signals-validation.yaml` (no external deps) to diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml new file mode 100644 index 00000000..f168d86d --- /dev/null +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml @@ -0,0 +1,203 @@ +# context-intelligence-query-validation.yaml +# +# RUNNABLE amplifier-digital-twin profile (base + provision + readiness). +# Proves the READ / EXECUTE-QUERIES seam: after events are logged to a real +# Context-Intelligence server, the CLI-loaded bundle's `graph_query` (Cypher) and +# `blob_read` (ci-blob:// URI) return REAL data AND a `source` provenance block that +# names the server that answered. Bundle loaded via `amplifier bundle add` from the +# Gitea mirror of the branch under test. NOT pytest/artefacts. +# +# TOPOLOGY +# host: a real Context-Intelligence server (Neo4j + API) reachable from the DTU at the +# Incus host-gateway IP (e.g. http://10.67.103.1:18001). +# DTU: amplifier + the branch bundle (full context-intelligence behavior). The read +# tools resolve (server_url, api_key) via ToolConfigResolver: +# explicit read-config -> hook `destinations` -> env AMPLIFIER_CONTEXT_INTELLIGENCE_*. +# Here the single-server env path is used, so the read side reuses the hook's +# destination as a read fallback (provenance origin = "destination"). +# +# HOW THE READ TOOLS ARE INVOKED (important, empirically discovered) +# `graph_query` / `blob_read` are NOT mounted as top-level tools in a plain `amplifier run` +# session, and `amplifier tool invoke` cannot drive them (the context-intelligence behavior +# has no orchestrator by itself -> "Configuration must specify session.orchestrator"). +# The real, shipped read path is the `context-intelligence:graph-analyst` AGENT: a session +# delegates to it, and the agent calls graph_query / blob_read in its sub-session. The +# validation step below drives exactly that path and captures the tool result (rows + the +# `source` provenance block) from the agent's response. +# +# HOW TO RUN +# # 1. stand up the server; 2. log some events to it first (e.g. run the write profile, or +# # just run one session in THIS DTU -- the hook writes as we read). Then: +# export GH_TOKEN=$(gh auth token) +# amplifier-digital-twin launch \ +# .amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml \ +# --name ci-query \ +# --var gitea_host=http://localhost:10110 \ +# --var server_url=http://10.67.103.1:18001 \ +# --var server_token= \ +# --var workspace=ci-query +# +# WHAT IS PROVEN +# STRUCTURAL (readiness gates, deterministic, no LLM): +# * bundle installed via the CLI, resolved to the MIRROR; graph-analyst agent present; +# the query-tool module (graph_query + blob_read) is in the loaded bundle. +# * the server is reachable from the DTU (neo4j_connected:true). +# BEHAVIOURAL (real session, real key; exec steps below): +# * graph_query returns >=1 row for a workspace-filtered Cypher query, and the tool +# result's `source` block names the server URL that answered. +# * blob_read resolves a ci-blob:// URI to real content, and its `source` block likewise +# names the answering server. + +name: context-intelligence-query-validation +description: > + Runnable proof of the read/execute-queries seam. Loads the branch bundle via the Amplifier + CLI from a Gitea mirror, logs events to a real Context-Intelligence server, then drives + graph_query (Cypher) and blob_read (ci-blob://) through the graph-analyst agent and proves + real rows/content come back WITH a source provenance block naming the answering server. + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + services: + - name: anthropic + key_env: ANTHROPIC_API_KEY + - name: github + key_env: GH_TOKEN + +provision: + setup_cmds: + - apt-get update && apt-get install -y git curl python3 python3-venv jq + + - curl -LsSf https://astral.sh/uv/install.sh | sh + + - | + if [ -n "${GH_TOKEN:-}" ]; then + echo "machine github.com login x-token-auth password $GH_TOKEN" > /root/.netrc + chmod 600 /root/.netrc + git config --global credential.helper 'store' + fi + + - | + git config --global \ + url."${gitea_host}/microsoft/amplifier-bundle-context-intelligence".insteadOf \ + "https://github.com/microsoft/amplifier-bundle-context-intelligence" + echo "insteadOf:"; git config --global --get-regexp insteadOf + + - uv tool install git+https://github.com/microsoft/amplifier@main + + - | + mkdir -p /root/.amplifier + cat > /root/.amplifier/settings.yaml << 'EOF' + config: + providers: + - module: provider-anthropic + source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main + config: + api_key_env: ANTHROPIC_API_KEY + EOF + + - | + amplifier bundle add \ + git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ + --app + + # Wire the read tools (and the write hook) to the server via the single-server env path. + - | + cat >> /etc/environment << ENVEOF + PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL=${server_url} + AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY=${server_token} + AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace} + ENVEOF + echo 'export PATH="/root/.local/bin:$PATH"' >> /root/.bashrc + echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL=${server_url}" >> /root/.bashrc + echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY=${server_token}" >> /root/.bashrc + echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace}" >> /root/.bashrc + + - amplifier --version + +readiness: + - name: amplifier-usable + command: "amplifier --version" + + - name: bundle-loaded-from-mirror + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + git -C "$C" remote -v | grep -q "${gitea_host}/microsoft/amplifier-bundle-context-intelligence" + && git -C "$C" log -1 --pretty=%s | grep -qi "docs/contributing-and-pr-template" + && echo "ready: bundle resolved from mirror @ branch snapshot" + + - name: graph-analyst-and-query-tools-loaded + # The read path is the graph-analyst agent; its module wiring must include the query tool + # (graph_query + blob_read) from the loaded (mirror) bundle. + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + test -f "$C/agents/graph-analyst.md" + && test -f "$C/modules/tool-context-intelligence-query/amplifier_module_tool_context_intelligence_query/graph_query_tool.py" + && test -f "$C/modules/tool-context-intelligence-query/amplifier_module_tool_context_intelligence_query/blob_read_tool.py" + && echo "ready: graph-analyst + graph_query + blob_read present in loaded bundle" + + - name: server-reachable-from-dtu + command: > + . /etc/environment; + curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" + | python3 -c "import sys,json;d=json.load(sys.stdin);assert d['status']=='ok' and d['neo4j_connected'];print('ready: server reachable, neo4j_connected')" + +validation_cmds: + # ---- BEHAVIOURAL step 0: ensure the server has events for this workspace to read back ---- + - name: B0-log-events-first + command: | + set -a; . /etc/environment; set +a + mkdir -p /root/ci-query && cd /root/ci-query + amplifier run "Say hello in exactly one word." --output-format json > /root/q0-run.json 2>/root/q0-run.err || cat /root/q0-run.err + sleep 3 + curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" \ + | jq "[.completed_sessions[]?|select(.workspace==\"$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE\")|{workspace, events_processed}]" + + # ---- BEHAVIOURAL step 1: graph_query rows + source provenance (via graph-analyst) ---- + - name: B1-graph-query-rows-and-provenance + command: | + set -a; . /etc/environment; set +a + cd /root/ci-query + WS="$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE" + PROMPT="Delegate to context-intelligence:graph-analyst to run this Cypher verbatim: MATCH (n) WHERE n.workspace = \"$WS\" RETURN labels(n) AS labels, count(n) AS c ORDER BY c DESC LIMIT 5 -- then quote the returned rows AND the full 'source' provenance block from the tool result." + amplifier run "$PROMPT" --output-format json-trace > /root/q1-trace.json 2>/root/q1.err || true + echo "== graph_query result (rows + source) extracted from the agent trace ==" + python3 - <<'PY' + import json + raw=open('/root/q1-trace.json').read(); i=raw.find('{'); d=json.loads(raw[i:]) + print("status:", d.get("status")) + for t in d.get("execution_trace",[]): + if t.get("type")=="tool_call" and t.get("tool")=="delegate": + out=t.get("result",{}); s=out if isinstance(out,str) else json.dumps(out) + print(s[:2000]) + print("FINAL:", (d.get("response") or "")[:600]) + PY + echo "^ PASS iff >=1 row returned AND source.url names $AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL" + + # ---- BEHAVIOURAL step 2: blob_read resolves a ci-blob:// URI + names the server ---- + - name: B2-blob-read-and-provenance + command: | + set -a; . /etc/environment; set +a + cd /root/ci-query + WS="$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE" + # discover a real ci-blob:// URI for this workspace directly from the server + BLOB=$(curl -sf -X POST "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/cypher" \ + -H "Authorization: Bearer $AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY" -H 'Content-Type: application/json' \ + -d "{\"query\":\"MATCH (n) WHERE n.workspace=\\\"$WS\\\" AND n.data CONTAINS \\\"ci-blob://\\\" WITH n.data AS d LIMIT 1 RETURN d\"}" \ + | python3 -c "import sys,json,re;print((re.search(r'ci-blob://[^\"\\\\]+',json.dumps(json.load(sys.stdin))) or [''])[0] if True else '')" 2>/dev/null) + echo "discovered blob: $BLOB" + PROMPT="Use blob_read (delegate to context-intelligence:graph-analyst) to read this blob URI and report the full 'source' provenance block plus its top-level keys: $BLOB" + amplifier run "$PROMPT" --output-format json-trace > /root/q2-trace.json 2>/root/q2.err || true + python3 - <<'PY' + import json + raw=open('/root/q2-trace.json').read(); i=raw.find('{'); d=json.loads(raw[i:]) + print("status:", d.get("status")) + for t in d.get("execution_trace",[]): + if t.get("type")=="tool_call" and t.get("tool")=="delegate": + out=t.get("result",{}); s=out if isinstance(out,str) else json.dumps(out) + print(s[:1800]) + PY + echo "^ PASS iff blob content keys returned AND source.url names $AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL" diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml new file mode 100644 index 00000000..4e85e630 --- /dev/null +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml @@ -0,0 +1,192 @@ +# context-intelligence-write-fanout-validation.yaml +# +# RUNNABLE amplifier-digital-twin profile (base + provision + readiness). +# Proves the WRITE FAN-OUT seam: the context-intelligence logging hook, loaded via the +# Amplifier CLI from a Gitea mirror, dispatches every session's events to TWO real +# Context-Intelligence servers configured as two `destinations`. This profile only +# OBSERVES the existing hook fan-out behaviour -- it does NOT modify the hook code. +# NOT pytest/artefacts. +# +# TOPOLOGY +# host: TWO real Context-Intelligence servers (each Neo4j + API), stood up via docker +# compose on distinct ports, reachable from the DTU at the Incus host-gateway IP +# (e.g. http://10.67.103.1:18001 and http://10.67.103.1:18002). +# DTU: amplifier + the branch bundle; the logging hook's config carries TWO `destinations` +# (srv_a, srv_b), each include:["**"], so a single session fans out to both. +# +# HOW THE TWO DESTINATIONS ARE INJECTED (important, load-bearing finding) +# The hook reads fan-out `destinations` from ITS MODULE CONFIG in the logging behavior +# YAML. The documented "project .amplifier/settings.yaml -> overrides.hook-context- +# intelligence.config.destinations" path DOES NOT WORK on this amplifier-foundation build: +# the configurator overlay states "Phase 1 covers agents, context, skills. tools/config +# overrides reserved for v1.1" -- so a project/app config override for the hook is SILENTLY +# IGNORED (verified: with only the project override set, NEITHER server received events). +# Until config-overrides land, the working injection is to write the two destinations +# directly into the loaded bundle's logging-behavior hook config. This profile does that +# deterministically in provision (patch_destinations.py) AFTER `amplifier bundle add`, so +# the CLI-loaded hook actually resolves two destinations. The env single-destination path +# (context_intelligence_server_url/api_key) only ever yields ONE destination and cannot +# express fan-out -- hence the config patch. +# +# HOW TO RUN +# # stand up TWO servers on the host (server repo docker-compose.yml; distinct ports + +# # data dirs + project names); confirm both: +# # curl -sf http://10.67.103.1:18001/status ; curl -sf http://10.67.103.1:18002/status +# export GH_TOKEN=$(gh auth token) +# amplifier-digital-twin launch \ +# .amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml \ +# --name ci-write-fanout \ +# --var gitea_host=http://localhost:10110 \ +# --var server_a_url=http://10.67.103.1:18001 --var server_a_token= \ +# --var server_b_url=http://10.67.103.1:18002 --var server_b_token= \ +# --var workspace=ci-write-fanout +# +# WHAT IS PROVEN +# STRUCTURAL (readiness gates, deterministic, no LLM): +# * bundle installed via the CLI, resolved to the MIRROR; logging hook present. +# * BOTH servers reachable from the DTU (neo4j_connected:true). +# * the loaded hook config resolves exactly TWO destinations (srv_a, srv_b), both include **. +# BEHAVIOURAL (real session, real key; exec step below): +# * ONE real `amplifier run` session fans out: BOTH servers' /status list the session +# under this workspace with events_processed>0, AND a Cypher count on EACH server +# returns >=1 node tagged workspace=. + +name: context-intelligence-write-fanout-validation +description: > + Runnable proof of the write fan-out seam. Loads the branch bundle via the Amplifier CLI + from a Gitea mirror, injects TWO real server destinations into the loaded logging hook + config, runs a single session, and proves BOTH servers received the events + (workspace-tagged) via /status and a per-server Cypher count. Observes existing hook + fan-out; does not modify hook code. + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + services: + - name: anthropic + key_env: ANTHROPIC_API_KEY + - name: github + key_env: GH_TOKEN + +provision: + setup_cmds: + - apt-get update && apt-get install -y git curl python3 python3-venv python3-yaml jq + + # Deterministic patcher: inject two fan-out destinations into the LOADED logging-behavior + # hook config (written inline so the profile is self-contained). See header note for why + # the project/app config-override path is a no-op on this foundation build. + - | + cat > /root/patch_destinations.py << 'PYEOF' + import glob, sys, yaml + a_url, a_tok, b_url, b_tok = sys.argv[1:5] + c = glob.glob('/root/.amplifier/cache/amplifier-bundle-context-intelligence-*')[0] + f = c + '/behaviors/context-intelligence-logging.yaml' + d = yaml.safe_load(open(f)) + cfg = d['hooks'][0]['config'] + cfg['destinations'] = { + 'srv_a': {'url': a_url, 'api_key': a_tok, 'include': ['**']}, + 'srv_b': {'url': b_url, 'api_key': b_tok, 'include': ['**']}, + } + yaml.safe_dump(d, open(f, 'w'), sort_keys=False) + print('patched', f, '-> destinations:', list(cfg['destinations'])) + PYEOF + + - curl -LsSf https://astral.sh/uv/install.sh | sh + + - | + if [ -n "${GH_TOKEN:-}" ]; then + echo "machine github.com login x-token-auth password $GH_TOKEN" > /root/.netrc + chmod 600 /root/.netrc + git config --global credential.helper 'store' + fi + + - | + git config --global \ + url."${gitea_host}/microsoft/amplifier-bundle-context-intelligence".insteadOf \ + "https://github.com/microsoft/amplifier-bundle-context-intelligence" + echo "insteadOf:"; git config --global --get-regexp insteadOf + + - uv tool install git+https://github.com/microsoft/amplifier@main + + - | + mkdir -p /root/.amplifier + cat > /root/.amplifier/settings.yaml << 'EOF' + config: + providers: + - module: provider-anthropic + source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main + config: + api_key_env: ANTHROPIC_API_KEY + EOF + + - | + amplifier bundle add \ + git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ + --app + + # Inject the two destinations into the loaded hook config (see header note on why the + # project/app config-override path is a no-op on this foundation build). + - | + python3 /root/patch_destinations.py \ + "${server_a_url}" "${server_a_token}" \ + "${server_b_url}" "${server_b_token}" + + - | + cat >> /etc/environment << ENVEOF + PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace} + SRV_A_URL=${server_a_url} + SRV_A_TOKEN=${server_a_token} + SRV_B_URL=${server_b_url} + SRV_B_TOKEN=${server_b_token} + ENVEOF + echo 'export PATH="/root/.local/bin:$PATH"' >> /root/.bashrc + echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace}" >> /root/.bashrc + + - amplifier --version + +readiness: + - name: amplifier-usable + command: "amplifier --version" + + - name: bundle-loaded-from-mirror + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + git -C "$C" remote -v | grep -q "${gitea_host}/microsoft/amplifier-bundle-context-intelligence" + && git -C "$C" log -1 --pretty=%s | grep -qi "docs/contributing-and-pr-template" + && echo "ready: bundle resolved from mirror @ branch snapshot" + + - name: two-destinations-in-loaded-hook-config + # Deterministic: the LOADED logging-behavior hook config must declare exactly srv_a + srv_b. + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + python3 -c "import yaml;d=yaml.safe_load(open('$C/behaviors/context-intelligence-logging.yaml'));dest=d['hooks'][0]['config']['destinations'];assert set(dest)=={'srv_a','srv_b'},dest;assert all(dest[k]['include']==['**'] for k in dest);print('ready: 2 destinations (srv_a, srv_b) in loaded hook config, both include **')" + + - name: both-servers-reachable + command: > + . /etc/environment; + curl -sf "$SRV_A_URL/status" | python3 -c "import sys,json;assert json.load(sys.stdin)['neo4j_connected']" && + curl -sf "$SRV_B_URL/status" | python3 -c "import sys,json;assert json.load(sys.stdin)['neo4j_connected']" && + echo "ready: both servers reachable, neo4j_connected" + +validation_cmds: + # ---- BEHAVIOURAL: run ONE session, prove BOTH servers received the events ---- + - name: B1-fanout-both-received + command: | + # export CI env so the child amplifier process + hook see it + set -a; . /etc/environment; set +a + cd /root + echo "== emit events through a real session ==" + amplifier run "Say hello in exactly one word." --output-format json > /root/fanout-run.json 2>/root/fanout-run.err || cat /root/fanout-run.err + sleep 3 + for pair in "A:$SRV_A_URL:$SRV_A_TOKEN" "B:$SRV_B_URL:$SRV_B_TOKEN"; do + NAME=${pair%%:*}; REST=${pair#*:}; URL=${REST%:*}; TOK=${REST##*:} + echo "== server $NAME ($URL): completed sessions for workspace=$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE ==" + curl -sf "$URL/status" | jq -c "[.completed_sessions[]?|select(.workspace==\"$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE\")|{workspace, events_processed}]" + echo "== server $NAME Cypher node count for workspace ==" + curl -sf -X POST "$URL/cypher" -H "Authorization: Bearer $TOK" -H 'Content-Type: application/json' \ + -d "{\"query\":\"MATCH (n) WHERE n.workspace = \\\"$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE\\\" RETURN count(n) AS n\"}" | jq . + done + echo "^ PASS iff BOTH servers show the workspace session with events_processed>0 AND node count >= 1" diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml new file mode 100644 index 00000000..84edea0a --- /dev/null +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml @@ -0,0 +1,164 @@ +# context-intelligence-write-server-validation.yaml +# +# RUNNABLE amplifier-digital-twin profile (base + provision + readiness). +# Proves the WRITE-TO-A-SINGLE-SERVER seam: the context-intelligence logging hook, +# loaded via the Amplifier CLI from a Gitea mirror of the branch under test, POSTs +# session events to ONE real Context-Intelligence server. NOT pytest/artefacts. +# +# TOPOLOGY +# host: a real Context-Intelligence server (Neo4j + API) stood up via docker compose, +# reachable from the DTU at the Incus host-gateway IP (e.g. http://10.67.103.1:18001). +# DTU: amplifier + the branch bundle (full context-intelligence behavior = design + logging), +# with the hook configured (legacy single-destination path) to dispatch to that server, +# tagged with an explicit workspace so events are independently queryable. +# +# HOW TO RUN +# # 1. stand up the server on the host (see server repo microsoft/amplifier-context-intelligence +# # docker-compose.yml -> start.sh generates credentials.yaml + prints the API token once). +# # Bind it to the Incus gateway IP so the container can reach it, and confirm: +# # curl -sf http://10.67.103.1:18001/status +# export GH_TOKEN=$(gh auth token) +# amplifier-digital-twin launch \ +# .amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml \ +# --name ci-write-single \ +# --var gitea_host=http://localhost:10110 \ +# --var server_url=http://10.67.103.1:18001 \ +# --var server_token= \ +# --var workspace=ci-write-single +# +# WHAT IS PROVEN +# STRUCTURAL (deterministic; readiness gates, run on launch, no LLM): +# * bundle installed via `amplifier bundle add ... --app` resolved to the MIRROR. +# * the logging hook is present in the loaded bundle. +# * the server is reachable from the DTU and reports neo4j_connected:true. +# BEHAVIOURAL (real session, real Anthropic key; the exec step below): +# * a real `amplifier run` session emits events; the hook POSTs them to the server; +# the server's /status lists the session under this workspace with events_processed>0, +# AND a Cypher query (/cypher) returns >=1 node tagged workspace=. +# +# NOTE ON write-to-FILE: local-JSONL capture is already covered by +# context-intelligence-bundle-smoke-test.yaml -- not duplicated here. + +name: context-intelligence-write-server-validation +description: > + Runnable proof of the write-to-a-single-server seam. Loads the branch bundle via the + Amplifier CLI from a Gitea mirror, wires the logging hook to ONE real Context-Intelligence + server, runs a real session, and proves the server received the events (tagged with the + workspace) via /status and a Cypher query. + +base: + image: ubuntu:24.04 + +passthrough: + allow_external: true + services: + - name: anthropic + key_env: ANTHROPIC_API_KEY + - name: github + key_env: GH_TOKEN + +provision: + setup_cmds: + - apt-get update && apt-get install -y git curl python3 python3-venv jq + + - curl -LsSf https://astral.sh/uv/install.sh | sh + + - | + if [ -n "${GH_TOKEN:-}" ]; then + echo "machine github.com login x-token-auth password $GH_TOKEN" > /root/.netrc + chmod 600 /root/.netrc + git config --global credential.helper 'store' + fi + + # Narrow rewrite: only the context-intelligence bundle self-references -> mirror. + - | + git config --global \ + url."${gitea_host}/microsoft/amplifier-bundle-context-intelligence".insteadOf \ + "https://github.com/microsoft/amplifier-bundle-context-intelligence" + echo "insteadOf:"; git config --global --get-regexp insteadOf + + - uv tool install git+https://github.com/microsoft/amplifier@main + + - | + mkdir -p /root/.amplifier + cat > /root/.amplifier/settings.yaml << 'EOF' + config: + providers: + - module: provider-anthropic + source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main + config: + api_key_env: ANTHROPIC_API_KEY + EOF + + # Load the branch bundle THROUGH THE CLI (redirected to the mirror). FULL behavior + # => design (read/query tools) + logging hook (write). + - | + amplifier bundle add \ + git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ + --app + + # Wire the logging hook to the SINGLE server via the legacy single-destination env path + # (context_intelligence_server_url/api_key + workspace). This synthesizes one "default" + # destination matching all sessions -- the shipped single-server path. + - | + cat >> /etc/environment << ENVEOF + PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL=${server_url} + AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY=${server_token} + AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace} + ENVEOF + cat >> /root/.bashrc << 'BRC' + export PATH="/root/.local/bin:$PATH" + BRC + echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL=${server_url}" >> /root/.bashrc + echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY=${server_token}" >> /root/.bashrc + echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace}" >> /root/.bashrc + + - amplifier --version + +readiness: + - name: amplifier-usable + command: "amplifier --version" + + - name: bundle-loaded-from-mirror + command: > + C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); + git -C "$C" remote -v | grep -q "${gitea_host}/microsoft/amplifier-bundle-context-intelligence" + && git -C "$C" log -1 --pretty=%s | grep -qi "docs/contributing-and-pr-template" + && echo "ready: bundle resolved from mirror @ branch snapshot" + + - name: logging-hook-present + command: > + amplifier bundle show context-intelligence-behavior 2>/dev/null + | grep -q "hook-context-intelligence" + && echo "ready: logging hook loaded via CLI" + + - name: server-reachable-from-dtu + command: > + . /etc/environment; + curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" + | python3 -c "import sys,json;d=json.load(sys.stdin);assert d['status']=='ok' and d['neo4j_connected'];print('ready: server reachable, neo4j_connected')" + +validation_cmds: + # ---- BEHAVIOURAL: run a real session, then prove the server received the events ---- + # Run via: amplifier-digital-twin exec -- bash -lc 'bash /root/run_write.sh' + - name: B1-emit-and-prove-received + command: | + # IMPORTANT: export the CI env so the child `amplifier` process (and its hook) see it. + set -a; . /etc/environment; set +a + mkdir -p /root/ci-write && cd /root/ci-write + echo "== emit events through a real session ==" + amplifier run "Say hello in exactly one word." --output-format json > /root/write-run.json 2>/root/write-run.err || cat /root/write-run.err + sleep 3 + echo "== server /status: sessions under workspace=$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE ==" + curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" \ + | jq '{active_sessions, sessions: [.sessions[]|{session_id, workspace, events_processed}], completed: [.completed_sessions[]?|{session_id, workspace, events_processed}]}' + echo "== Cypher: count nodes tagged with this workspace ==" + # NOTE: this server's /cypher does NOT accept a `parameters` map -> inline the literal + # workspace (safe; we control it). Use WHERE n.workspace = "". + curl -sf -X POST "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/cypher" \ + -H "Authorization: Bearer $AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d "{\"query\":\"MATCH (n) WHERE n.workspace = \\\"$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE\\\" RETURN count(n) AS n\"}" \ + | jq . + echo "^ PASS iff a session with this workspace has events_processed>0 AND node count >= 1" From 7c5b30d269e1fd49177c5905324773f7500cec9e Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 20:55:50 +0000 Subject: [PATCH 10/12] refactor(dtu): rename validation_cmds -> manual_validation_steps (honest field name) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit amplifier-digital-twin never executes `validation_cmds` β€” it is an unknown field the launcher silently ignores (only base/provision/readiness run). The name implied automated execution the runner does not provide. Renamed to manual_validation_steps across all 8 profiles that carried it (+ the 2 stale references in the profiles README), so the field reads as what it is: documented steps a human runs via `amplifier-digital-twin exec -- …`. Command bodies preserved verbatim; readiness gates untouched; all YAML still parses. (Council-flagged; machete pass.) πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .amplifier/digital-twin-universe/profiles/README.md | 4 ++-- .../profiles/context-intelligence-bundle-smoke-test.yaml | 2 +- ...ntext-intelligence-contributes-migration-validation.yaml | 4 ++-- .../context-intelligence-mode-activation-validation.yaml | 4 ++-- .../profiles/context-intelligence-query-validation.yaml | 2 +- .../context-intelligence-redesigned-mode-validation.yaml | 6 +++--- .../profiles/context-intelligence-signals-validation.yaml | 2 +- .../context-intelligence-write-fanout-validation.yaml | 2 +- .../context-intelligence-write-server-validation.yaml | 2 +- 9 files changed, 14 insertions(+), 14 deletions(-) diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index 8fa6dea7..5b7c6349 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -63,7 +63,7 @@ seams without those; that is the point of an end-to-end test. ### How the mode profiles gate (runnable schema) `amplifier-digital-twin` auto-runs a profile's **`provision`** and **`readiness`** on launch -(it ignores unknown keys like `validation_cmds`). So the three mode profiles put their +(it ignores unknown keys like `manual_validation_steps`). So the three mode profiles put their **deterministic structural proofs in `readiness`** β€” they gate the launch and fail it if the CLI-loaded bundle is wrong. Each mode profile's `readiness` proves, for real: @@ -79,7 +79,7 @@ CLI-loaded bundle is wrong. Each mode profile's `readiness` proves, for real: **Behavioural** activation β€” the real offβ†’onβ†’off round-trip (`/mode context-intelligence` β†’ `[context-intelligence]>` β†’ `/mode off`) β€” is proven with a real Anthropic session and is -documented in each profile's `validation_cmds` as a **reproducible manual step** +documented in each profile's `manual_validation_steps` as a **reproducible manual step** (`amplifier-digital-twin exec -- …`), because it needs a live PTY session. **Honest limitation:** the *runtime-mounted set while the mode is active* (exactly which diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml index 4af00076..e9721d0f 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-bundle-smoke-test.yaml @@ -101,7 +101,7 @@ readiness: - name: library-check command: "/opt/venv/bin/python3 -c \"from context_intelligence.client import AsyncCIClient; print('ready')\"" -validation_cmds: +manual_validation_steps: # Install the stub event contributor fixture module from the local bundle - name: install-stub-module command: | diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml index 0d8eba43..e33bb32c 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-contributes-migration-validation.yaml @@ -11,7 +11,7 @@ # --name ci-contributes-migration \ # --var gitea_host=http://localhost:10110 # -# `readiness` RUNS ON LAUNCH (structural proof, no LLM). `validation_cmds` documents +# `readiness` RUNS ON LAUNCH (structural proof, no LLM). `manual_validation_steps` documents # the full check-suite; run each via `amplifier-digital-twin exec -- `. # # WHAT THIS PROFILE FOCUSES ON (the contributes.agents migration): @@ -117,7 +117,7 @@ readiness: && ! echo "$OUT" | grep -q "context-intelligence-tool-designer" && echo "ready: specialists gated (contributes.agents), baseline intact" -validation_cmds: +manual_validation_steps: # ---- STRUCTURAL ---- - name: S1-inactive-baseline-and-gating command: | diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml index 1f95153b..0d4e0194 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-mode-activation-validation.yaml @@ -12,7 +12,7 @@ # --name ci-mode-activation \ # --var gitea_host=http://localhost:10110 # -# `readiness` RUNS ON LAUNCH (structural proof). `validation_cmds` documents the full +# `readiness` RUNS ON LAUNCH (structural proof). `manual_validation_steps` documents the full # suite; run each via `amplifier-digital-twin exec -- ` (behavioural steps # need an interactive TTY -- see notes). # @@ -127,7 +127,7 @@ readiness: && grep -q "context-intelligence-tool-designer" "$M" && echo "ready: full gated surface (3 context + 3 skills + 2 agents) declared" -validation_cmds: +manual_validation_steps: # ---- STRUCTURAL ---- - name: S1-inactive-baseline command: | diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml index f168d86d..c7ce0fdb 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-query-validation.yaml @@ -145,7 +145,7 @@ readiness: curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" | python3 -c "import sys,json;d=json.load(sys.stdin);assert d['status']=='ok' and d['neo4j_connected'];print('ready: server reachable, neo4j_connected')" -validation_cmds: +manual_validation_steps: # ---- BEHAVIOURAL step 0: ensure the server has events for this workspace to read back ---- - name: B0-log-events-first command: | diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml index 638f07ac..73b44e45 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-redesigned-mode-validation.yaml @@ -14,10 +14,10 @@ # # The `readiness` checks below RUN ON LAUNCH and gate the environment: they prove # (deterministically, no LLM) that the bundle loaded from the mirror and that the -# mode-gating surface is correct when the mode is INACTIVE. The `validation_cmds` +# mode-gating surface is correct when the mode is INACTIVE. The `manual_validation_steps` # block documents the full structural + behavioural check-suite; run each step via # amplifier-digital-twin exec -- -# (`validation_cmds` is not auto-executed by launch -- it is the documented, +# (`manual_validation_steps` is not auto-executed by launch -- it is the documented, # reproducible evidence procedure, incl. the interactive-TTY behavioural steps). # # WHAT IS PROVABLE VIA THE CLI (discovered empirically -- see notes per step): @@ -136,7 +136,7 @@ readiness: && ! echo "$OUT" | grep -q "context-intelligence-tool-designer" && echo "ready: baseline agents present, specialists gated when mode inactive" -validation_cmds: +manual_validation_steps: # ---- STRUCTURAL (deterministic; run via `amplifier-digital-twin exec -- ...`) ---- - name: S1-mirror-provenance command: | diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml index dc123ebd..e8aa83a8 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-signals-validation.yaml @@ -83,7 +83,7 @@ readiness: /opt/venv/bin/python3 -c "from context_intelligence.signals import score_session; print('ready')" -validation_cmds: +manual_validation_steps: # A: All public symbols importable - name: A-library-import command: | diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml index 4e85e630..9fc1990a 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml @@ -171,7 +171,7 @@ readiness: curl -sf "$SRV_B_URL/status" | python3 -c "import sys,json;assert json.load(sys.stdin)['neo4j_connected']" && echo "ready: both servers reachable, neo4j_connected" -validation_cmds: +manual_validation_steps: # ---- BEHAVIOURAL: run ONE session, prove BOTH servers received the events ---- - name: B1-fanout-both-received command: | diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml index 84edea0a..02b23709 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml @@ -139,7 +139,7 @@ readiness: curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" | python3 -c "import sys,json;d=json.load(sys.stdin);assert d['status']=='ok' and d['neo4j_connected'];print('ready: server reachable, neo4j_connected')" -validation_cmds: +manual_validation_steps: # ---- BEHAVIOURAL: run a real session, then prove the server received the events ---- # Run via: amplifier-digital-twin exec -- bash -lc 'bash /root/run_write.sh' - name: B1-emit-and-prove-received From 0e06cce303411a55877cb0cf532f035c89ebc3e7 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 21:10:55 +0000 Subject: [PATCH 11/12] docs(dtu): retract the incorrect config-override finding (was my misconfiguration) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The earlier "destinations override is a no-op / reserved for v1.1" note was wrong: the repro used a bare list of destinations instead of the bundle README's named-map format (overrides.hook-context-intelligence.config.destinations keyed by name, api_key via ${VAR} from ~/.amplifier/keys.env), and conflated the app-cli settings.yaml override (documented + working) with the foundation configurator's unrelated "reserved for v1.1" comment. Corrected the README working-order note accordingly; the fan-out profile's direct-injection is flagged as a test workaround to redo with the correct config + re-verify, not evidence of a framework limitation. Support issue closed as invalid. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .../digital-twin-universe/profiles/README.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index 5b7c6349..5453572f 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -206,12 +206,17 @@ tools (see the main `README.md` Β§"read side"). For multi-source, the connectabl **5 real rows** and `blob_read` resolved a real `ci-blob://…__raw` URI (44 KB, 8 top-level keys), each carrying the `source` provenance block naming the answering server (`{name:default, origin:destination, url:http://…:18001}`). - Instances + server stacks then destroyed. **Honest findings from the run:** (1) the documented - project-`settings.yaml` `overrides.hook-context-intelligence.config.destinations` path is a - **no-op on the current amplifier-foundation build** (tools/config overrides "reserved for - v1.1"), so the fan-out profile injects destinations directly into the loaded hook config; (2) - `graph_query`/`blob_read` are **not** top-level tools in a plain session β€” the shipped read - path is the `context-intelligence:graph-analyst` agent, which is what the query profile drives. + Instances + server stacks then destroyed. **Notes from the run:** (1) `destinations` must + follow the bundle README's format β€” a **named map** under + `overrides.hook-context-intelligence.config.destinations` (keyed by destination name), with + each `api_key` referenced as a `${VAR}` from `~/.amplifier/keys.env` (see the main `README.md` + Β§"Server forwarding β€” `destinations`"). An earlier run used a bare *list* of destinations + (wrong shape) and so did not dispatch; the fan-out profile's direct-injection of destinations + is a **test workaround for that misconfiguration and should be redone with the correct + named-map `settings.yaml` config + re-verified** β€” it is *not* evidence of a framework + limitation. (2) `graph_query`/`blob_read` are **not** top-level tools in a plain session β€” the + shipped read path is the `context-intelligence:graph-analyst` agent, which is what the query + profile drives. - **Runtime-green is per-launch**, per the AGENTS.md rule β€” capture the run evidence (real request/response, provenance, fail-loud on a down/500/timeout) when you exercise a seam. Start with `context-intelligence-signals-validation.yaml` (no external deps) to From 960e17bb95738f816339e35b88e83a35a31362f5 Mon Sep 17 00:00:00 2001 From: colombod Date: Mon, 13 Jul 2026 21:47:38 +0000 Subject: [PATCH 12/12] test(dtu): configure write/fanout destinations the documented way + re-verify MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the direct-hook-config-injection workaround in the write-server and write-fanout profiles with the documented user path: `~/.amplifier/settings.yaml` overrides.hook-context-intelligence.config.destinations as a NAMED MAP (keyed by destination name), each api_key a ${VAR} from `~/.amplifier/keys.env` β€” exactly as the bundle README specifies. patch_destinations.py removed from both. Re-verified live against fresh isolated CI server stacks (docker compose), through the documented config only: single-server -> server A received the session (events_processed:22), server B received nothing; fan-out -> one session, two named destinations -> BOTH servers independently received it (each events_processed:22, same workspace). Also fixed a readiness gate that used a folded YAML scalar (collapsed a heredoc -> exit 2); now literal-block, 5/5 ready:True on relaunch. Root cause of the earlier "no-op" (support issue closed as invalid): it used a project-scope .amplifier/settings.yaml (routes through the foundation configurator overlay, not yet applied for hook/tool config) AND a bare-list destinations shape. The user-level ~/.amplifier/settings.yaml named-map is the implemented, working path (app-cli get_config_overrides). README evidence updated. πŸ€– Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com> --- .../digital-twin-universe/profiles/README.md | 26 +-- ...-intelligence-write-fanout-validation.yaml | 162 ++++++++++-------- ...-intelligence-write-server-validation.yaml | 142 ++++++++++----- 3 files changed, 209 insertions(+), 121 deletions(-) diff --git a/.amplifier/digital-twin-universe/profiles/README.md b/.amplifier/digital-twin-universe/profiles/README.md index 5453572f..fe4dbccf 100644 --- a/.amplifier/digital-twin-universe/profiles/README.md +++ b/.amplifier/digital-twin-universe/profiles/README.md @@ -206,17 +206,21 @@ tools (see the main `README.md` Β§"read side"). For multi-source, the connectabl **5 real rows** and `blob_read` resolved a real `ci-blob://…__raw` URI (44 KB, 8 top-level keys), each carrying the `source` provenance block naming the answering server (`{name:default, origin:destination, url:http://…:18001}`). - Instances + server stacks then destroyed. **Notes from the run:** (1) `destinations` must - follow the bundle README's format β€” a **named map** under - `overrides.hook-context-intelligence.config.destinations` (keyed by destination name), with - each `api_key` referenced as a `${VAR}` from `~/.amplifier/keys.env` (see the main `README.md` - Β§"Server forwarding β€” `destinations`"). An earlier run used a bare *list* of destinations - (wrong shape) and so did not dispatch; the fan-out profile's direct-injection of destinations - is a **test workaround for that misconfiguration and should be redone with the correct - named-map `settings.yaml` config + re-verified** β€” it is *not* evidence of a framework - limitation. (2) `graph_query`/`blob_read` are **not** top-level tools in a plain session β€” the - shipped read path is the `context-intelligence:graph-analyst` agent, which is what the query - profile drives. + Instances + server stacks then destroyed. These two write profiles configure destinations via + the **documented user path only** β€” `~/.amplifier/settings.yaml` + `overrides.hook-context-intelligence.config.destinations` as a **named map** (keyed by + destination name), each `api_key` a `${VAR}` from `~/.amplifier/keys.env` (see the main + `README.md` Β§"Server forwarding β€” `destinations`"). Re-verified live through that path (no + direct-config-injection workaround): **single-server** β†’ server A received the session + (`events_processed:22`), server B correctly received nothing; **fan-out** β†’ one session, two + named destinations β†’ **both** servers independently received it (each `events_processed:22`, + same workspace). **Notes:** (1) the earlier "no-op" was a mis-test β€” it used a *project-scope* + `.amplifier/settings.yaml` (which routes through the foundation configurator overlay, not yet + applied for hook/tool `config`) **and** a bare-*list* shape; the **user-level** + `~/.amplifier/settings.yaml` named-map is the implemented, working path (app-cli + `get_config_overrides`). No framework limitation β€” the support issue was closed as invalid. + (2) `graph_query`/`blob_read` are **not** top-level tools in a plain session β€” the shipped read + path is the `context-intelligence:graph-analyst` agent, which the query profile drives. - **Runtime-green is per-launch**, per the AGENTS.md rule β€” capture the run evidence (real request/response, provenance, fail-loud on a down/500/timeout) when you exercise a seam. Start with `context-intelligence-signals-validation.yaml` (no external deps) to diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml index 9fc1990a..f8fd75f7 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-fanout-validation.yaml @@ -7,26 +7,29 @@ # OBSERVES the existing hook fan-out behaviour -- it does NOT modify the hook code. # NOT pytest/artefacts. # +# CONFIG PATH = THE DOCUMENTED USER PATH (README "Server forwarding -- destinations") +# `destinations` is a NAMED MAP under +# overrides.hook-context-intelligence.config.destinations in ~/.amplifier/settings.yaml +# Two named destinations here -- `main` (server A) and `team` (server B) -- each with an +# api_key referenced as ${VAR} from ~/.amplifier/keys.env (MAIN_CI_KEY / TEAM_CI_KEY). +# The app-cli loads keys.env at startup and expands ${VAR} before it reaches the hook; +# each destination whose include-pattern matches the session's working dir receives the +# events -- true fan-out. This is the real user path -- NO direct patch of the loaded +# hook-config YAML, NO deprecated env scalars. +# +# (Historical note: an earlier version patched the cached bundle's logging-behavior YAML +# because a PROJECT-scope `.amplifier/settings.yaml` override was a no-op. That project +# overlay is the amplifier-foundation configurator overlay "reserved for v1.1" -- the WRONG +# path. The USER-level ~/.amplifier/settings.yaml `overrides..config` path is +# applied by the app-cli's get_config_overrides() to providers/tools/hooks alike, and IS +# implemented. This profile uses only that -- the patch workaround is fully removed.) +# # TOPOLOGY # host: TWO real Context-Intelligence servers (each Neo4j + API), stood up via docker # compose on distinct ports, reachable from the DTU at the Incus host-gateway IP # (e.g. http://10.67.103.1:18001 and http://10.67.103.1:18002). -# DTU: amplifier + the branch bundle; the logging hook's config carries TWO `destinations` -# (srv_a, srv_b), each include:["**"], so a single session fans out to both. -# -# HOW THE TWO DESTINATIONS ARE INJECTED (important, load-bearing finding) -# The hook reads fan-out `destinations` from ITS MODULE CONFIG in the logging behavior -# YAML. The documented "project .amplifier/settings.yaml -> overrides.hook-context- -# intelligence.config.destinations" path DOES NOT WORK on this amplifier-foundation build: -# the configurator overlay states "Phase 1 covers agents, context, skills. tools/config -# overrides reserved for v1.1" -- so a project/app config override for the hook is SILENTLY -# IGNORED (verified: with only the project override set, NEITHER server received events). -# Until config-overrides land, the working injection is to write the two destinations -# directly into the loaded bundle's logging-behavior hook config. This profile does that -# deterministically in provision (patch_destinations.py) AFTER `amplifier bundle add`, so -# the CLI-loaded hook actually resolves two destinations. The env single-destination path -# (context_intelligence_server_url/api_key) only ever yields ONE destination and cannot -# express fan-out -- hence the config patch. +# DTU: amplifier + the branch bundle; the hook config carries TWO named destinations, +# each include:["**"], so a single session fans out to both. # # HOW TO RUN # # stand up TWO servers on the host (server repo docker-compose.yml; distinct ports + @@ -44,8 +47,9 @@ # WHAT IS PROVEN # STRUCTURAL (readiness gates, deterministic, no LLM): # * bundle installed via the CLI, resolved to the MIRROR; logging hook present. +# * ~/.amplifier/settings.yaml declares exactly TWO named destinations (main, team), both +# include ["**"], each api_key a ${VAR} reference (never a literal); keys.env supplies both. # * BOTH servers reachable from the DTU (neo4j_connected:true). -# * the loaded hook config resolves exactly TWO destinations (srv_a, srv_b), both include **. # BEHAVIOURAL (real session, real key; exec step below): # * ONE real `amplifier run` session fans out: BOTH servers' /status list the session # under this workspace with events_processed>0, AND a Cypher count on EACH server @@ -53,11 +57,11 @@ name: context-intelligence-write-fanout-validation description: > - Runnable proof of the write fan-out seam. Loads the branch bundle via the Amplifier CLI - from a Gitea mirror, injects TWO real server destinations into the loaded logging hook - config, runs a single session, and proves BOTH servers received the events - (workspace-tagged) via /status and a per-server Cypher count. Observes existing hook - fan-out; does not modify hook code. + Runnable proof of the write fan-out seam via the DOCUMENTED config path (TWO named-map + destinations in ~/.amplifier/settings.yaml + ${VAR} api_keys from ~/.amplifier/keys.env). + Loads the branch bundle via the Amplifier CLI from a Gitea mirror, runs a single session, + and proves BOTH servers received the events (workspace-tagged) via /status and a per-server + Cypher count. Observes existing hook fan-out; does not modify hook code or patch loaded YAML. base: image: ubuntu:24.04 @@ -74,25 +78,6 @@ provision: setup_cmds: - apt-get update && apt-get install -y git curl python3 python3-venv python3-yaml jq - # Deterministic patcher: inject two fan-out destinations into the LOADED logging-behavior - # hook config (written inline so the profile is self-contained). See header note for why - # the project/app config-override path is a no-op on this foundation build. - - | - cat > /root/patch_destinations.py << 'PYEOF' - import glob, sys, yaml - a_url, a_tok, b_url, b_tok = sys.argv[1:5] - c = glob.glob('/root/.amplifier/cache/amplifier-bundle-context-intelligence-*')[0] - f = c + '/behaviors/context-intelligence-logging.yaml' - d = yaml.safe_load(open(f)) - cfg = d['hooks'][0]['config'] - cfg['destinations'] = { - 'srv_a': {'url': a_url, 'api_key': a_tok, 'include': ['**']}, - 'srv_b': {'url': b_url, 'api_key': b_tok, 'include': ['**']}, - } - yaml.safe_dump(d, open(f, 'w'), sort_keys=False) - print('patched', f, '-> destinations:', list(cfg['destinations'])) - PYEOF - - curl -LsSf https://astral.sh/uv/install.sh | sh - | @@ -110,40 +95,63 @@ provision: - uv tool install git+https://github.com/microsoft/amplifier@main + # ---- keys.env: SECRETS ONLY. Two bearer tokens, named MAIN_CI_KEY + TEAM_CI_KEY. + # DTU substitutes ${server_a_token}/${server_b_token} (--vars) to literals BEFORE the + # container shell runs; the quoted heredoc leaves them untouched thereafter. - | mkdir -p /root/.amplifier - cat > /root/.amplifier/settings.yaml << 'EOF' + cat > /root/.amplifier/keys.env << 'KEYSEOF' + MAIN_CI_KEY=${server_a_token} + TEAM_CI_KEY=${server_b_token} + KEYSEOF + chmod 600 /root/.amplifier/keys.env + + # ---- settings.yaml: the DOCUMENTED config. Provider + TWO named-map destinations. + # api_key values are ${VAR} REFERENCES (literal text, NOT secrets) -- MAIN_CI_KEY/ + # TEAM_CI_KEY are not DTU --vars, so DTU leaves them intact and the quoted heredoc keeps + # the shell from expanding them; the app-cli expands them from keys.env before mount. + # ${server_a_url}/${server_b_url}/${workspace} ARE --vars -> DTU substitutes them here. + - | + cat > /root/.amplifier/settings.yaml << 'SETTINGSEOF' config: providers: - module: provider-anthropic source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main config: api_key_env: ANTHROPIC_API_KEY - EOF + overrides: + hook-context-intelligence: + config: + workspace: "${workspace}" + destinations: + main: + url: "${server_a_url}" + api_key: "${MAIN_CI_KEY}" + include: ["**"] + team: + url: "${server_b_url}" + api_key: "${TEAM_CI_KEY}" + include: ["**"] + SETTINGSEOF + echo "== settings.yaml written =="; cat /root/.amplifier/settings.yaml - | amplifier bundle add \ git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ --app - # Inject the two destinations into the loaded hook config (see header note on why the - # project/app config-override path is a no-op on this foundation build). + # ---- verification-only endpoints (NOT hook config): lets the readiness/manual steps curl + # both servers. The hook is configured EXCLUSIVELY by settings.yaml + keys.env above. - | - python3 /root/patch_destinations.py \ - "${server_a_url}" "${server_a_token}" \ - "${server_b_url}" "${server_b_token}" - - - | - cat >> /etc/environment << ENVEOF - PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin - AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace} + cat > /root/ci-verify.env << 'VERIFYEOF' SRV_A_URL=${server_a_url} SRV_A_TOKEN=${server_a_token} SRV_B_URL=${server_b_url} SRV_B_TOKEN=${server_b_token} - ENVEOF + WORKSPACE=${workspace} + VERIFYEOF echo 'export PATH="/root/.local/bin:$PATH"' >> /root/.bashrc - echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace}" >> /root/.bashrc + echo 'PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' >> /etc/environment - amplifier --version @@ -158,35 +166,53 @@ readiness: && git -C "$C" log -1 --pretty=%s | grep -qi "docs/contributing-and-pr-template" && echo "ready: bundle resolved from mirror @ branch snapshot" - - name: two-destinations-in-loaded-hook-config - # Deterministic: the LOADED logging-behavior hook config must declare exactly srv_a + srv_b. - command: > - C=$(find /root/.amplifier/cache -maxdepth 1 -type d -name 'amplifier-bundle-context-intelligence-*' | head -1); - python3 -c "import yaml;d=yaml.safe_load(open('$C/behaviors/context-intelligence-logging.yaml'));dest=d['hooks'][0]['config']['destinations'];assert set(dest)=={'srv_a','srv_b'},dest;assert all(dest[k]['include']==['**'] for k in dest);print('ready: 2 destinations (srv_a, srv_b) in loaded hook config, both include **')" + - name: two-destinations-in-user-settings + # Deterministic proof of the DOCUMENTED config: ~/.amplifier/settings.yaml declares + # exactly two named destinations (main, team), both include ["**"], each api_key a ${VAR} + # reference (NOT a literal), and ~/.amplifier/keys.env supplies both referenced vars. + # NOTE: literal block scalar (|) -- a folded scalar (>) would collapse the heredoc newlines. + command: | + python3 - <<'PY' + import re, yaml + s = yaml.safe_load(open('/root/.amplifier/settings.yaml')) + dest = s['overrides']['hook-context-intelligence']['config']['destinations'] + assert set(dest) == {'main', 'team'}, dest + env = open('/root/.amplifier/keys.env').read() + for name in ('main', 'team'): + api = dest[name]['api_key'] + m = re.fullmatch(r'\$\{([A-Za-z_][A-Za-z0-9_]*)\}', api.strip()) + assert m, f'{name}.api_key must be a ${{VAR}} reference, got literal: {api!r}' + var = m.group(1) + assert re.search(rf'(?m)^{var}=', env), f'{var} missing from keys.env' + assert dest[name]['include'] == ['**'], dest[name] + print('ready: 2 named destinations (main, team); api_keys are ${VAR} refs; keys.env supplies both') + PY - name: both-servers-reachable command: > - . /etc/environment; + . /root/ci-verify.env; curl -sf "$SRV_A_URL/status" | python3 -c "import sys,json;assert json.load(sys.stdin)['neo4j_connected']" && curl -sf "$SRV_B_URL/status" | python3 -c "import sys,json;assert json.load(sys.stdin)['neo4j_connected']" && echo "ready: both servers reachable, neo4j_connected" manual_validation_steps: - # ---- BEHAVIOURAL: run ONE session, prove BOTH servers received the events ---- + # ---- BEHAVIOURAL: run ONE session (hook configured ONLY via settings.yaml + keys.env), + # prove BOTH servers received the events. - name: B1-fanout-both-received command: | - # export CI env so the child amplifier process + hook see it - set -a; . /etc/environment; set +a - cd /root + # No AMPLIFIER_CONTEXT_INTELLIGENCE_* env exports -- the hook is driven purely by the + # documented ~/.amplifier/settings.yaml + keys.env. ci-verify.env is only for the curls. + . /root/ci-verify.env + mkdir -p /root/ci-fanout && cd /root/ci-fanout echo "== emit events through a real session ==" amplifier run "Say hello in exactly one word." --output-format json > /root/fanout-run.json 2>/root/fanout-run.err || cat /root/fanout-run.err sleep 3 for pair in "A:$SRV_A_URL:$SRV_A_TOKEN" "B:$SRV_B_URL:$SRV_B_TOKEN"; do NAME=${pair%%:*}; REST=${pair#*:}; URL=${REST%:*}; TOK=${REST##*:} - echo "== server $NAME ($URL): completed sessions for workspace=$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE ==" - curl -sf "$URL/status" | jq -c "[.completed_sessions[]?|select(.workspace==\"$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE\")|{workspace, events_processed}]" + echo "== server $NAME ($URL): completed sessions for workspace=$WORKSPACE ==" + curl -sf "$URL/status" | jq -c "[.completed_sessions[]?|select(.workspace==\"$WORKSPACE\")|{workspace, events_processed}]" echo "== server $NAME Cypher node count for workspace ==" curl -sf -X POST "$URL/cypher" -H "Authorization: Bearer $TOK" -H 'Content-Type: application/json' \ - -d "{\"query\":\"MATCH (n) WHERE n.workspace = \\\"$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE\\\" RETURN count(n) AS n\"}" | jq . + -d "{\"query\":\"MATCH (n) WHERE n.workspace = \\\"$WORKSPACE\\\" RETURN count(n) AS n\"}" | jq . done echo "^ PASS iff BOTH servers show the workspace session with events_processed>0 AND node count >= 1" diff --git a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml index 02b23709..4f684761 100644 --- a/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml +++ b/.amplifier/digital-twin-universe/profiles/context-intelligence-write-server-validation.yaml @@ -5,15 +5,28 @@ # loaded via the Amplifier CLI from a Gitea mirror of the branch under test, POSTs # session events to ONE real Context-Intelligence server. NOT pytest/artefacts. # +# CONFIG PATH = THE DOCUMENTED USER PATH (README "Getting started" + "Server forwarding") +# The hook's `destinations` is a NAMED MAP under +# overrides.hook-context-intelligence.config.destinations in ~/.amplifier/settings.yaml +# with each destination's api_key referenced as ${VAR} from ~/.amplifier/keys.env +# (loaded automatically by the app-cli at startup; ${VAR} expanded before it reaches the +# hook). This is the real user path -- NOT the deprecated single-server env scalars, and +# NOT any direct patch of the bundle's loaded hook-config YAML. +# +# (Historical note: an earlier version of this profile wrote a PROJECT-scope +# `.amplifier/settings.yaml` and, when that was a no-op, patched the cached bundle YAML. +# The project-scope overlay is the amplifier-foundation configurator overlay that is +# "reserved for v1.1" -- the WRONG path. The USER-level ~/.amplifier/settings.yaml +# `overrides..config` path is applied by the app-cli's get_config_overrides() +# to providers/tools/hooks alike, and IS implemented. This profile uses only that.) +# # TOPOLOGY # host: a real Context-Intelligence server (Neo4j + API) stood up via docker compose, # reachable from the DTU at the Incus host-gateway IP (e.g. http://10.67.103.1:18001). -# DTU: amplifier + the branch bundle (full context-intelligence behavior = design + logging), -# with the hook configured (legacy single-destination path) to dispatch to that server, -# tagged with an explicit workspace so events are independently queryable. +# DTU: amplifier + the branch bundle (full context-intelligence behavior = design + logging). # # HOW TO RUN -# # 1. stand up the server on the host (see server repo microsoft/amplifier-context-intelligence +# # 1. stand up the server on the host (server repo microsoft/amplifier-context-intelligence # # docker-compose.yml -> start.sh generates credentials.yaml + prints the API token once). # # Bind it to the Incus gateway IP so the container can reach it, and confirm: # # curl -sf http://10.67.103.1:18001/status @@ -29,22 +42,25 @@ # WHAT IS PROVEN # STRUCTURAL (deterministic; readiness gates, run on launch, no LLM): # * bundle installed via `amplifier bundle add ... --app` resolved to the MIRROR. -# * the logging hook is present in the loaded bundle. +# * ~/.amplifier/settings.yaml declares the named 'main' destination with api_key as a +# ${VAR} reference (never a literal), and ~/.amplifier/keys.env supplies that var. # * the server is reachable from the DTU and reports neo4j_connected:true. # BEHAVIOURAL (real session, real Anthropic key; the exec step below): -# * a real `amplifier run` session emits events; the hook POSTs them to the server; -# the server's /status lists the session under this workspace with events_processed>0, -# AND a Cypher query (/cypher) returns >=1 node tagged workspace=. +# * a real `amplifier run` session emits events; the hook (configured ONLY via the +# documented settings.yaml + keys.env) POSTs them to the server; the server's /status +# lists the session under this workspace with events_processed>0, AND a Cypher query +# (/cypher) returns >=1 node tagged workspace=. # # NOTE ON write-to-FILE: local-JSONL capture is already covered by # context-intelligence-bundle-smoke-test.yaml -- not duplicated here. name: context-intelligence-write-server-validation description: > - Runnable proof of the write-to-a-single-server seam. Loads the branch bundle via the - Amplifier CLI from a Gitea mirror, wires the logging hook to ONE real Context-Intelligence - server, runs a real session, and proves the server received the events (tagged with the - workspace) via /status and a Cypher query. + Runnable proof of the write-to-a-single-server seam via the DOCUMENTED config path + (named-map destinations in ~/.amplifier/settings.yaml + ${VAR} api_key from + ~/.amplifier/keys.env). Loads the branch bundle via the Amplifier CLI from a Gitea mirror, + runs a real session, and proves the server received the events (workspace-tagged) via + /status and a Cypher query. base: image: ubuntu:24.04 @@ -59,7 +75,7 @@ passthrough: provision: setup_cmds: - - apt-get update && apt-get install -y git curl python3 python3-venv jq + - apt-get update && apt-get install -y git curl python3 python3-venv python3-yaml jq - curl -LsSf https://astral.sh/uv/install.sh | sh @@ -79,40 +95,59 @@ provision: - uv tool install git+https://github.com/microsoft/amplifier@main + # ---- keys.env: SECRETS ONLY (never committed). The bearer token, named MAIN_CI_KEY. + # DTU substitutes ${server_token} (a --var) to the literal token BEFORE the container + # shell runs; the quoted heredoc leaves it untouched thereafter. - | mkdir -p /root/.amplifier - cat > /root/.amplifier/settings.yaml << 'EOF' + cat > /root/.amplifier/keys.env << 'KEYSEOF' + MAIN_CI_KEY=${server_token} + KEYSEOF + chmod 600 /root/.amplifier/keys.env + + # ---- settings.yaml: the DOCUMENTED config. Provider + the NAMED-MAP destination. + # api_key is the ${VAR} REFERENCE (literal text, NOT the secret) -- MAIN_CI_KEY is not a + # DTU --var, so DTU leaves ${MAIN_CI_KEY} intact; the quoted heredoc keeps the shell from + # expanding it; the app-cli expands it from keys.env before it reaches the hook. + # ${server_url}/${workspace} ARE --vars, so DTU substitutes them to literals here. + - | + cat > /root/.amplifier/settings.yaml << 'SETTINGSEOF' config: providers: - module: provider-anthropic source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main config: api_key_env: ANTHROPIC_API_KEY - EOF + overrides: + hook-context-intelligence: + config: + workspace: "${workspace}" + destinations: + main: + url: "${server_url}" + api_key: "${MAIN_CI_KEY}" + include: ["**"] + SETTINGSEOF + echo "== settings.yaml written =="; cat /root/.amplifier/settings.yaml # Load the branch bundle THROUGH THE CLI (redirected to the mirror). FULL behavior - # => design (read/query tools) + logging hook (write). + # => design (read/query tools) + logging hook (write). bundle add --app merges its + # registry entry into the settings.yaml written above (preserving config + overrides). - | amplifier bundle add \ git+https://github.com/microsoft/amplifier-bundle-context-intelligence@main#subdirectory=behaviors/context-intelligence.yaml \ --app - # Wire the logging hook to the SINGLE server via the legacy single-destination env path - # (context_intelligence_server_url/api_key + workspace). This synthesizes one "default" - # destination matching all sessions -- the shipped single-server path. + # ---- verification-only endpoint file (NOT hook config): lets the readiness/manual steps + # curl the server. The hook is configured EXCLUSIVELY by settings.yaml + keys.env above. - | - cat >> /etc/environment << ENVEOF - PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin - AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL=${server_url} - AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY=${server_token} - AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace} - ENVEOF - cat >> /root/.bashrc << 'BRC' - export PATH="/root/.local/bin:$PATH" - BRC - echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL=${server_url}" >> /root/.bashrc - echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY=${server_token}" >> /root/.bashrc - echo "export AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE=${workspace}" >> /root/.bashrc + cat > /root/ci-verify.env << 'VERIFYEOF' + SERVER_URL=${server_url} + SERVER_TOKEN=${server_token} + WORKSPACE=${workspace} + VERIFYEOF + echo 'export PATH="/root/.local/bin:$PATH"' >> /root/.bashrc + echo 'PATH=/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' >> /etc/environment - amplifier --version @@ -133,32 +168,55 @@ readiness: | grep -q "hook-context-intelligence" && echo "ready: logging hook loaded via CLI" + - name: destination-configured-in-user-settings + # Deterministic proof of the DOCUMENTED config: ~/.amplifier/settings.yaml declares a + # single named destination 'main' whose api_key is a ${VAR} reference (NOT a literal), + # and ~/.amplifier/keys.env supplies that referenced var. No LLM. + # NOTE: literal block scalar (|) -- a folded scalar (>) would collapse the heredoc newlines. + command: | + python3 - <<'PY' + import re, yaml, sys + s = yaml.safe_load(open('/root/.amplifier/settings.yaml')) + dest = s['overrides']['hook-context-intelligence']['config']['destinations'] + assert set(dest) == {'main'}, dest + api = dest['main']['api_key'] + m = re.fullmatch(r'\$\{([A-Za-z_][A-Za-z0-9_]*)\}', api.strip()) + assert m, f'api_key must be a ${{VAR}} reference, got literal: {api!r}' + var = m.group(1) + env = open('/root/.amplifier/keys.env').read() + assert re.search(rf'(?m)^{var}=', env), f'{var} missing from keys.env' + assert dest['main']['include'] == ['**'] + print(f'ready: destination main -> api_key ${{{var}}} (referenced, not literal); keys.env supplies {var}') + PY + - name: server-reachable-from-dtu command: > - . /etc/environment; - curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" + . /root/ci-verify.env; + curl -sf "$SERVER_URL/status" | python3 -c "import sys,json;d=json.load(sys.stdin);assert d['status']=='ok' and d['neo4j_connected'];print('ready: server reachable, neo4j_connected')" manual_validation_steps: - # ---- BEHAVIOURAL: run a real session, then prove the server received the events ---- - # Run via: amplifier-digital-twin exec -- bash -lc 'bash /root/run_write.sh' + # ---- BEHAVIOURAL: run a real session (hook configured ONLY via settings.yaml + keys.env), + # then prove the server received the events. Run via: + # amplifier-digital-twin exec -- bash -lc 'bash /root/run_write.sh' - name: B1-emit-and-prove-received command: | - # IMPORTANT: export the CI env so the child `amplifier` process (and its hook) see it. - set -a; . /etc/environment; set +a + # No AMPLIFIER_CONTEXT_INTELLIGENCE_* env exports -- the hook is driven purely by the + # documented ~/.amplifier/settings.yaml + keys.env. ci-verify.env is only for the curls. + . /root/ci-verify.env mkdir -p /root/ci-write && cd /root/ci-write echo "== emit events through a real session ==" amplifier run "Say hello in exactly one word." --output-format json > /root/write-run.json 2>/root/write-run.err || cat /root/write-run.err sleep 3 - echo "== server /status: sessions under workspace=$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE ==" - curl -sf "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/status" \ + echo "== server /status: sessions under workspace=$WORKSPACE ==" + curl -sf "$SERVER_URL/status" \ | jq '{active_sessions, sessions: [.sessions[]|{session_id, workspace, events_processed}], completed: [.completed_sessions[]?|{session_id, workspace, events_processed}]}' echo "== Cypher: count nodes tagged with this workspace ==" # NOTE: this server's /cypher does NOT accept a `parameters` map -> inline the literal # workspace (safe; we control it). Use WHERE n.workspace = "". - curl -sf -X POST "$AMPLIFIER_CONTEXT_INTELLIGENCE_SERVER_URL/cypher" \ - -H "Authorization: Bearer $AMPLIFIER_CONTEXT_INTELLIGENCE_API_KEY" \ + curl -sf -X POST "$SERVER_URL/cypher" \ + -H "Authorization: Bearer $SERVER_TOKEN" \ -H 'Content-Type: application/json' \ - -d "{\"query\":\"MATCH (n) WHERE n.workspace = \\\"$AMPLIFIER_CONTEXT_INTELLIGENCE_WORKSPACE\\\" RETURN count(n) AS n\"}" \ + -d "{\"query\":\"MATCH (n) WHERE n.workspace = \\\"$WORKSPACE\\\" RETURN count(n) AS n\"}" \ | jq . echo "^ PASS iff a session with this workspace has events_processed>0 AND node count >= 1"