Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ README에서는 빠른 검토 경로만 유지하고, 세부 marker contract는
| telemetry/replay evidence가 충분한가? | telemetry replay gap, `runtime_history_seed_run_config_traceability`, `edgeenv_orchestrator_producer_lineage` | Runtime seed, `run_config`, EdgeEnv history, Orchestrator lineage가 Lab까지 보존됐는지 확인합니다. |
| 먼저 열어볼 operation risk가 있는가? | `Reviewer operation quick scan`, `raw_marker=reviewer_focus_operation_quick_scan`, `Orchestrator queue/deadline/fallback markers` | detailed row를 열기 전에 queue pressure, `max_total_queue_depth`, deadline miss, fallback count, Jetson/device-local identity를 먼저 봅니다. |
| 어떤 preserved run/path를 보는가? | `Runtime replay duration scope`, `Lab EdgeEnv preservation context`, `Jetson/device-local EdgeEnv preservation run`, `Jetson/device-local EdgeEnv preservation details` | replay duration과 `identity=jetson_device_local_preservation`, `path=device_local_starter` label을 빠르게 찾게 합니다. |
| optional AIGuard full-evidence source fixture는 무엇인가? | `runtime_intelligence_source_traceability_summary.md`, `source_traceability_alignment`, `aiguard_optional_present_source_artifact` | EdgeEnv mirror와 AIGuard optional-present fixture가 같은 read-only source artifact와 재생성 명령을 가리키는지 확인합니다. |
| 어떤 warning이 review context로만 남는가? | `AIGuard max queue raw-context traceability`, `AIGuard operation timeline evidence`, `AIGuard runtime operation anomalies`, `AIGuard remote dispatch event summary`, `Remote fallback starter evidence`, `production_remote_execution=false` | AIGuard/Orchestrator가 policy owner가 되지 않도록 deterministic warning과 remote starter evidence를 보존합니다. |

세부 marker contract는 [docs/portfolio/edgeenv_runtime_regression_lab_handoff.md](docs/portfolio/edgeenv_runtime_regression_lab_handoff.md)에 정리되어 있습니다.
Expand Down Expand Up @@ -281,6 +282,9 @@ bash scripts/smoke_runtime_intelligence_chain.sh \
해석하지 않습니다. 이어서 source traceability gate가 EdgeEnv mirror와
AIGuard optional-present fixture가 같은 source artifact와 명령을 가리키는지
확인합니다.
생성된 Lab report의 Review Path는 manifest/report artifact gate와 함께
`runtime_intelligence_source_traceability_summary.md`와
`source_traceability_alignment` marker도 가리킵니다.
EdgeEnv나 AIGuard가 Lab의 final deployment decision을 대체하지 않습니다.

## 현재 범위와 future work
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -537,6 +537,7 @@ in the handoff document linked below.
| Is telemetry/replay evidence complete enough? | telemetry replay gaps, `runtime_history_seed_run_config_traceability`, `edgeenv_orchestrator_producer_lineage` | Shows whether Runtime seed, `run_config`, EdgeEnv history, and Orchestrator lineage reached Lab. |
| Is there an operation risk worth opening first? | `Reviewer operation quick scan`, `raw_marker=reviewer_focus_operation_quick_scan`, `Orchestrator queue/deadline/fallback markers` | Surfaces queue pressure, `max_total_queue_depth`, deadline misses, fallback count, and Jetson/device-local identity before the detailed rows. |
| Which preserved run/path is being reviewed? | `Runtime replay duration scope`, `Lab EdgeEnv preservation context`, `Jetson/device-local EdgeEnv preservation run`, `Jetson/device-local EdgeEnv preservation details` | Keeps replay duration, `identity=jetson_device_local_preservation`, and `path=device_local_starter` easy to find. |
| Which optional AIGuard source fixture backs the full-evidence path? | `runtime_intelligence_source_traceability_summary.md`, `source_traceability_alignment`, `aiguard_optional_present_source_artifact` | Lets reviewers confirm the EdgeEnv mirror and AIGuard optional-present fixture point to the same read-only source artifact and reproduction command. |
| Which warnings remain review context only? | `AIGuard max queue raw-context traceability`, `AIGuard operation timeline evidence`, `AIGuard runtime operation anomalies`, `AIGuard remote dispatch event summary`, `Remote fallback starter evidence`, `production_remote_execution=false` | Preserves deterministic warning and remote starter evidence without making AIGuard/Orchestrator the policy owner. |

These rows do not turn Orchestrator, EdgeEnv, or AIGuard into final decision
Expand Down Expand Up @@ -581,7 +582,7 @@ bash scripts/smoke_runtime_intelligence_chain.sh \
--output-dir reports/runtime_intelligence_chain
```

The smoke gates the EdgeEnv handoff history fixture for preserved device-local Orchestrator `candidate_context.producer` lineage and checks that EdgeEnv-declared external AIGuard evidence requirements are satisfied by the bundled `guard_analysis`. It also carries a precomputed AIGuard handoff-alignment artifact so the EdgeEnv handoff summary and AIGuard producer-lineage raw context agree on the same `producer_lineage_guard_alignment_run_ids`. When the handoff declares `optional_aiguard_evidence_types`, the copied AIGuard alignment artifacts preserve them as `read_only_optional_guard_context` and exercise both missing-optional and present-optional paths without turning optional evidence into required evidence. When it also declares `optional_aiguard_source_traceability`, the bundle gate validates the `read_only_optional_source_traceability` source artifact path and AIGuard reproduction command without making EdgeEnv produce `guard_analysis`; the source traceability gate then checks that the EdgeEnv mirror and AIGuard optional-present fixture reference the same source artifact and command. This is an artifact integrity check only; EdgeEnv still owns comparability/regression evidence, AIGuard remains an optional deterministic evidence provider, and Lab still owns the deployment decision.
The smoke gates the EdgeEnv handoff history fixture for preserved device-local Orchestrator `candidate_context.producer` lineage and checks that EdgeEnv-declared external AIGuard evidence requirements are satisfied by the bundled `guard_analysis`. It also carries a precomputed AIGuard handoff-alignment artifact so the EdgeEnv handoff summary and AIGuard producer-lineage raw context agree on the same `producer_lineage_guard_alignment_run_ids`. When the handoff declares `optional_aiguard_evidence_types`, the copied AIGuard alignment artifacts preserve them as `read_only_optional_guard_context` and exercise both missing-optional and present-optional paths without turning optional evidence into required evidence. When it also declares `optional_aiguard_source_traceability`, the bundle gate validates the `read_only_optional_source_traceability` source artifact path and AIGuard reproduction command without making EdgeEnv produce `guard_analysis`; the source traceability gate then checks that the EdgeEnv mirror and AIGuard optional-present fixture reference the same source artifact and command. The generated Lab report Review Path points reviewers to `runtime_intelligence_source_traceability_summary.md` and its `source_traceability_alignment` marker alongside the manifest and report artifact gates. This is an artifact integrity check only; EdgeEnv still owns comparability/regression evidence, AIGuard remains an optional deterministic evidence provider, and Lab still owns the deployment decision.

The committed handoff smoke is documented in
[docs/portfolio/edgeenv_runtime_regression_lab_handoff.md](docs/portfolio/edgeenv_runtime_regression_lab_handoff.md)
Expand Down
6 changes: 6 additions & 0 deletions docs/portfolio/edgeenv_runtime_regression_lab_handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,11 @@ This second smoke uses committed lightweight artifacts to represent the cross-re
- The bundle manifest also declares `expected_report_markers`; Lab's gate requires the declared marker set to match the Lab-owned Runtime Intelligence report contract before the report is generated.
- The same gate can also consume `--edgeenv-handoff` to compare EdgeEnv producer-side `lab_bundle_alignment` metadata against Lab's bundle manifest contract.
- `scripts/check_runtime_intelligence_artifact_bundle.py` gates the generated report so required Runtime Intelligence rows, remote dispatch summary/boundary rows, and ownership text cannot disappear silently.
- The generated Runtime Intelligence report Review Path points reviewers to
`runtime_intelligence_source_traceability_summary.md` and
`source_traceability_alignment` alongside the manifest and report artifact
gates, so optional AIGuard source provenance remains visible from the
Lab-owned report.

Expected Lab behavior:

Expand Down Expand Up @@ -234,6 +239,7 @@ Expected Lab behavior:
`review_path_fast_path: readable Review Path fast path rendered`,
`review_path: Reviewer Focus -> Detailed Evidence Rows guidance validated`,
`review_path_artifact_gate_summary: artifact gate summary reference row validated`,
`review_path_source_traceability_summary: source traceability summary reference row validated`,
preserving the generated report reading order as reviewer navigation
evidence without making CI or AIGuard the report owner.
- The Jetson/device-local preservation row starts with `identity=jetson_device_local_preservation` and the device-local path marker such as `path=device_local_starter` when available, making the preserved Jetson EdgeEnv run easier to identify before reviewers inspect detailed queue/resource context.
Expand Down
4 changes: 3 additions & 1 deletion inferedgelab/report/html_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -647,8 +647,10 @@ def _runtime_intelligence_risk_summary_to_html(
"replay gap, preserved run/path, or deterministic warning evidence.</div>"
'<div class="review-step"><span class="review-step-index">3</span>'
"<strong>Artifact Gate Summary</strong>: cross-check "
"<code>runtime_intelligence_bundle_manifest_gate_summary.md</code> and "
"<code>runtime_intelligence_bundle_manifest_gate_summary.md</code>, "
"<code>runtime_intelligence_source_traceability_summary.md</code>, and "
"<code>runtime_anomaly_gate_summary.md</code> for "
"<code>source_traceability_alignment</code>, "
"<code>reviewer_path_gate</code>, "
"<code>reviewer_path_local_links</code>, and "
"<code>reviewer_path_anchor_fragments</code> marker preservation.</div>"
Expand Down
9 changes: 5 additions & 4 deletions inferedgelab/report/markdown_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -370,10 +370,11 @@ def _append_runtime_intelligence_risk_summary(
)
lines.append(
"| 3 | `Artifact Gate Summary` | Cross-check "
"`runtime_intelligence_bundle_manifest_gate_summary.md` and "
"`runtime_anomaly_gate_summary.md` for `reviewer_path_gate`, "
"`reviewer_path_local_links`, and `reviewer_path_anchor_fragments` "
"marker preservation. |"
"`runtime_intelligence_bundle_manifest_gate_summary.md`, "
"`runtime_intelligence_source_traceability_summary.md`, and "
"`runtime_anomaly_gate_summary.md` for `source_traceability_alignment`, "
"`reviewer_path_gate`, `reviewer_path_local_links`, and "
"`reviewer_path_anchor_fragments` marker preservation. |"
)
lines.append("")
if focus_rows:
Expand Down
9 changes: 9 additions & 0 deletions scripts/check_runtime_intelligence_artifact_bundle.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@
"| 3 | `Artifact Gate Summary` | Cross-check "
"`runtime_intelligence_bundle_manifest_gate_summary.md`"
),
"review_path_source_traceability_summary": (
"`runtime_intelligence_source_traceability_summary.md`"
),
"review_path_source_traceability_marker": "`source_traceability_alignment`",
"review_path_artifact_gate_summary_markers": (
"`reviewer_path_gate`, `reviewer_path_local_links`, and "
"`reviewer_path_anchor_fragments`"
Expand Down Expand Up @@ -211,6 +215,10 @@
"review_path_artifact_gate_summary_file": (
"runtime_intelligence_bundle_manifest_gate_summary.md"
),
"review_path_source_traceability_summary_file": (
"runtime_intelligence_source_traceability_summary.md"
),
"review_path_source_traceability_marker": "source_traceability_alignment",
"review_path_artifact_gate_summary_markers": "reviewer_path_anchor_fragments",
"review_path_scope": (
"only for comparable regression, telemetry/replay gaps, operation quick scan"
Expand Down Expand Up @@ -356,6 +364,7 @@
"review_path: Reviewer Focus -> Detailed Evidence Rows guidance validated",
"review_path_scope: comparable regression / telemetry replay / operation evidence preserved",
"review_path_artifact_gate_summary: artifact gate summary reference row validated",
"review_path_source_traceability_summary: source traceability summary reference row validated",
)


Expand Down
3 changes: 3 additions & 0 deletions scripts/check_runtime_intelligence_ci_artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,7 @@
"review_path: Reviewer Focus -> Detailed Evidence Rows guidance validated",
"review_path_scope: comparable regression / telemetry replay / operation evidence preserved",
"review_path_artifact_gate_summary: artifact gate summary reference row validated",
"review_path_source_traceability_summary: source traceability summary reference row validated",
)


Expand Down Expand Up @@ -304,6 +305,8 @@ def _validate_runtime_report(path: Path, errors: list[str]) -> None:
"Fast path: `Reviewer Focus` -> `Detailed Evidence Rows` only when a quick signal needs supporting evidence.",
"| Step | Open | Use it for |",
"| 3 | `Artifact Gate Summary` | Cross-check `runtime_intelligence_bundle_manifest_gate_summary.md`",
"`runtime_intelligence_source_traceability_summary.md`",
"`source_traceability_alignment`",
"`reviewer_path_gate`, `reviewer_path_local_links`, and `reviewer_path_anchor_fragments`",
"only for comparable regression, telemetry/replay gaps, operation quick scan",
"Runtime replay duration scope",
Expand Down
9 changes: 5 additions & 4 deletions tests/test_report_generators.py
Original file line number Diff line number Diff line change
Expand Up @@ -970,10 +970,11 @@ def test_generate_compare_markdown_summarizes_orchestrator_context_risk():
) in text
assert (
"| 3 | `Artifact Gate Summary` | Cross-check "
"`runtime_intelligence_bundle_manifest_gate_summary.md` and "
"`runtime_anomaly_gate_summary.md` for `reviewer_path_gate`, "
"`reviewer_path_local_links`, and `reviewer_path_anchor_fragments` "
"marker preservation. |"
"`runtime_intelligence_bundle_manifest_gate_summary.md`, "
"`runtime_intelligence_source_traceability_summary.md`, and "
"`runtime_anomaly_gate_summary.md` for `source_traceability_alignment`, "
"`reviewer_path_gate`, `reviewer_path_local_links`, and "
"`reviewer_path_anchor_fragments` marker preservation. |"
) in text
assert "### Reviewer Focus" in text
assert "| Focus | Quick signal | First read |" in text
Expand Down
7 changes: 7 additions & 0 deletions tests/test_runtime_intelligence_ci_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
"- review_path: Reviewer Focus -> Detailed Evidence Rows guidance validated",
"- review_path_scope: comparable regression / telemetry replay / operation evidence preserved",
"- review_path_artifact_gate_summary: artifact gate summary reference row validated",
"- review_path_source_traceability_summary: source traceability summary reference row validated",
]
) + "\n"
SOURCE_TRACEABILITY_SUMMARY = "\n".join(
Expand All @@ -46,6 +47,8 @@
"Fast path: `Reviewer Focus` -> `Detailed Evidence Rows` only when a quick signal needs supporting evidence.",
"| Step | Open | Use it for |",
"| 3 | `Artifact Gate Summary` | Cross-check `runtime_intelligence_bundle_manifest_gate_summary.md`",
"`runtime_intelligence_source_traceability_summary.md`",
"`source_traceability_alignment`",
"`reviewer_path_gate`, `reviewer_path_local_links`, and `reviewer_path_anchor_fragments`",
"only for comparable regression, telemetry/replay gaps, operation quick scan",
]
Expand Down Expand Up @@ -464,6 +467,10 @@ def test_runtime_intelligence_ci_artifact_gate_passes_for_expected_outputs(tmp_p
"review_path_artifact_gate_summary: artifact gate summary reference row validated"
in summary
)
assert (
"review_path_source_traceability_summary: source traceability summary reference row validated"
in summary
)
assert "## Validated AIGuard Optional Handoff Context" in summary
assert (
"aiguard_optional_context: read_only_optional_guard_context preserved"
Expand Down
4 changes: 4 additions & 0 deletions tests/test_runtime_intelligence_evidence_chain_smoke.py
Original file line number Diff line number Diff line change
Expand Up @@ -536,6 +536,8 @@ def test_compare_cmd_runtime_intelligence_chain_writes_markdown_and_html(
"| 3 | `Artifact Gate Summary` | Cross-check "
"`runtime_intelligence_bundle_manifest_gate_summary.md`"
) in markdown
assert "`runtime_intelligence_source_traceability_summary.md`" in markdown
assert "`source_traceability_alignment`" in markdown
assert "reviewer_path_anchor_fragments" in markdown
assert "### Reviewer Focus" in markdown
assert "| Focus | Quick signal | First read |" in markdown
Expand Down Expand Up @@ -645,6 +647,8 @@ def test_compare_cmd_runtime_intelligence_chain_writes_markdown_and_html(
assert "<strong>Detailed Evidence Rows</strong>: open only the rows needed" in html
assert "<strong>Artifact Gate Summary</strong>: cross-check" in html
assert "runtime_intelligence_bundle_manifest_gate_summary.md" in html
assert "runtime_intelligence_source_traceability_summary.md" in html
assert "source_traceability_alignment" in html
assert "reviewer_path_anchor_fragments" in html
assert "Lab remains the final deployment decision owner." in html
assert "Reviewer Focus" in html
Expand Down
7 changes: 7 additions & 0 deletions tests/test_runtime_intelligence_smoke_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,8 @@ def test_runtime_intelligence_smoke_script_runs_artifact_chain(tmp_path):
encoding="utf-8"
)
assert "Runtime replay duration scope" in runtime_summary
assert "runtime_intelligence_source_traceability_summary.md" in runtime_summary
assert "source_traceability_alignment" in runtime_summary
assert "short 96-frame-class replay (96 frames)" in runtime_summary
assert "class=short_96_frame_class, frames=96" in runtime_summary
assert "source=entrypoint_requested_frames" in runtime_summary
Expand Down Expand Up @@ -384,3 +386,8 @@ def test_runtime_intelligence_smoke_script_runs_artifact_chain(tmp_path):
in ci_summary
)
assert "duration_label: short 96-frame-class replay (96 frames)" in ci_summary
assert (
"review_path_source_traceability_summary: "
"source traceability summary reference row validated"
in ci_summary
)
Loading