Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,10 @@ bash scripts/smoke_runtime_intelligence_chain.sh \

이 smoke는 EdgeEnv가 선언한 external AIGuard evidence requirement가 bundled
`guard_analysis`로 충족되는지도 확인하는 artifact integrity check입니다.
또한 `optional_aiguard_source_traceability`가 있으면 Lab bundle gate가
`read_only_optional_source_traceability` source artifact path와 AIGuard
재생성 명령을 검증하되, EdgeEnv가 `guard_analysis`를 생성한다는 의미로
해석하지 않습니다.
EdgeEnv나 AIGuard가 Lab의 final deployment decision을 대체하지 않습니다.

## 현재 범위와 future work
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -581,7 +581,7 @@ bash scripts/smoke_runtime_intelligence_chain.sh \
--output-dir reports/runtime_intelligence_chain
```

The smoke gates the EdgeEnv handoff history fixture for preserved device-local Orchestrator `candidate_context.producer` lineage and checks that EdgeEnv-declared external AIGuard evidence requirements are satisfied by the bundled `guard_analysis`. It also carries a precomputed AIGuard handoff-alignment artifact so the EdgeEnv handoff summary and AIGuard producer-lineage raw context agree on the same `producer_lineage_guard_alignment_run_ids`. When the handoff declares `optional_aiguard_evidence_types`, the copied AIGuard alignment artifacts preserve them as `read_only_optional_guard_context` and exercise both missing-optional and present-optional paths without turning optional evidence into required evidence. This is an artifact integrity check only; EdgeEnv still owns comparability/regression evidence, AIGuard remains an optional deterministic evidence provider, and Lab still owns the deployment decision.
The smoke gates the EdgeEnv handoff history fixture for preserved device-local Orchestrator `candidate_context.producer` lineage and checks that EdgeEnv-declared external AIGuard evidence requirements are satisfied by the bundled `guard_analysis`. It also carries a precomputed AIGuard handoff-alignment artifact so the EdgeEnv handoff summary and AIGuard producer-lineage raw context agree on the same `producer_lineage_guard_alignment_run_ids`. When the handoff declares `optional_aiguard_evidence_types`, the copied AIGuard alignment artifacts preserve them as `read_only_optional_guard_context` and exercise both missing-optional and present-optional paths without turning optional evidence into required evidence. When it also declares `optional_aiguard_source_traceability`, the bundle gate validates the `read_only_optional_source_traceability` source artifact path and AIGuard reproduction command without making EdgeEnv produce `guard_analysis`. This is an artifact integrity check only; EdgeEnv still owns comparability/regression evidence, AIGuard remains an optional deterministic evidence provider, and Lab still owns the deployment decision.

The committed handoff smoke is documented in
[docs/portfolio/edgeenv_runtime_regression_lab_handoff.md](docs/portfolio/edgeenv_runtime_regression_lab_handoff.md)
Expand Down
4 changes: 4 additions & 0 deletions docs/ci/runtime_intelligence_gitlab_artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@ The initial gate is conservative:
validates the known stale-drop labels (`stale_frame_risk` and
`edgeenv_orchestrator_stale_drop_summary`) as optional context that remains
separate from the required AIGuard evidence set
- if the handoff declares `optional_aiguard_source_traceability`, the bundle
gate validates the `read_only_optional_source_traceability` AIGuard source
artifact path and reproduction command as read-only metadata, without making
EdgeEnv produce `guard_analysis`
- the EdgeEnv handoff `runtime_telemetry_history` file must exist and preserve
the EdgeEnv history schema, telemetry coverage summary, and Runtime history
seed ownership markers
Expand Down
5 changes: 5 additions & 0 deletions docs/portfolio/edgeenv_runtime_regression_lab_handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,11 @@ This second smoke uses committed lightweight artifacts to represent the cross-re
`edgeenv_orchestrator_stale_drop_summary`) as optional context only. Those
labels must remain separate from the required Guard evidence set and do not
become a Lab deployment-decision requirement.
- If the handoff also declares `optional_aiguard_source_traceability`, Lab
validates the `read_only_optional_source_traceability` source artifact path
and AIGuard reproduction command as read-only metadata. This links the
EdgeEnv handoff to the AIGuard optional-present full-evidence fixture without
making EdgeEnv produce `guard_analysis`.
- `examples/runtime_intelligence_chain/runtime_telemetry_history.json` is the EdgeEnv producer-side telemetry history artifact referenced by the handoff manifest. It includes a missing-telemetry run as an evidence gap and preserves Orchestrator context on that entry without turning Orchestrator into a regression or deployment decision owner.
- Orchestrator context is preserved inside the EdgeEnv regression artifact as `orchestrator_operation_context`.
- AIGuard deterministic queue/thermal and task-event rollup evidence is passed as a precomputed `guard_analysis` artifact that mirrors the AIGuard producer-side diagnosis v1 evidence shape.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,32 @@
"stale_frame_risk",
"edgeenv_orchestrator_stale_drop_summary"
],
"optional_aiguard_source_traceability": {
"context_role": "read_only_optional_source_traceability",
"edgeenv_does_not_generate_guard_analysis": true,
"lab_is_final_decision_owner": true,
"optional_present_source_artifact": {
"repository": "InferEdgeAIGuard",
"path": "examples/runtime_intelligence/aiguard_runtime_operation_guard_analysis_optional_stale_drop.json",
"schema_version": "inferedge-aiguard-diagnosis-v1",
"role": "aiguard-optional-stale-drop-full-evidence-source",
"context_role": "read_only_cross_repo_traceability",
"reproduction_command": [
"python",
"-m",
"inferedge_aiguard.cli",
"build-runtime-intelligence-optional-stale-drop",
"--edgeenv-regression",
"examples/runtime_intelligence/edgeenv_runtime_regression_with_optional_stale_drop_context.json",
"--remote-dispatch",
"examples/runtime_intelligence/remote_dispatch_fallback_recovered_result.json",
"--orchestration-summary",
"examples/runtime_intelligence/orchestrator_multi_workload_sustained_summary.json",
"--save-json",
"examples/runtime_intelligence/aiguard_runtime_operation_guard_analysis_optional_stale_drop.json"
]
}
},
"expected_report_markers": [
"Runtime Intelligence Risk Summary",
"Runtime replay duration scope",
Expand Down
120 changes: 120 additions & 0 deletions scripts/check_runtime_intelligence_bundle_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,42 @@
"stale_frame_risk",
"edgeenv_orchestrator_stale_drop_summary",
}
OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_CONTEXT_ROLE = (
"read_only_optional_source_traceability"
)
OPTIONAL_AIGUARD_SOURCE_ARTIFACT = {
"repository": "InferEdgeAIGuard",
"path": (
"examples/runtime_intelligence/"
"aiguard_runtime_operation_guard_analysis_optional_stale_drop.json"
),
"schema_version": "inferedge-aiguard-diagnosis-v1",
"role": "aiguard-optional-stale-drop-full-evidence-source",
"context_role": "read_only_cross_repo_traceability",
"reproduction_command": [
"python",
"-m",
"inferedge_aiguard.cli",
"build-runtime-intelligence-optional-stale-drop",
"--edgeenv-regression",
(
"examples/runtime_intelligence/"
"edgeenv_runtime_regression_with_optional_stale_drop_context.json"
),
"--remote-dispatch",
"examples/runtime_intelligence/remote_dispatch_fallback_recovered_result.json",
"--orchestration-summary",
(
"examples/runtime_intelligence/"
"orchestrator_multi_workload_sustained_summary.json"
),
"--save-json",
(
"examples/runtime_intelligence/"
"aiguard_runtime_operation_guard_analysis_optional_stale_drop.json"
),
],
}
REQUIRED_GUARD_EVIDENCE_FIELDS = {
"type",
"metric_name",
Expand Down Expand Up @@ -205,6 +241,9 @@
EDGEENV_HANDOFF_OPTIONAL_AIGUARD_SUMMARY_MARKER = (
"edgeenv_handoff: optional AIGuard evidence types declared"
)
EDGEENV_HANDOFF_OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_SUMMARY_MARKER = (
"edgeenv_handoff: optional AIGuard source traceability declared"
)


def _load_json(path: Path, label: str) -> dict[str, Any]:
Expand Down Expand Up @@ -485,6 +524,8 @@ def _validate_edgeenv_handoff_alignment(
"required AIGuard evidence types",
)

_validate_optional_aiguard_source_traceability(alignment, errors)

expected_report_markers = alignment.get("expected_report_markers")
_record(
isinstance(expected_report_markers, list),
Expand Down Expand Up @@ -1887,6 +1928,54 @@ def _validate_external_aiguard_evidence_alignment(
)


def _validate_optional_aiguard_source_traceability(
alignment: dict[str, Any],
errors: list[str],
) -> None:
traceability = alignment.get("optional_aiguard_source_traceability")
if traceability is None:
return
_record(
isinstance(traceability, dict),
errors,
"EdgeEnv handoff lab_bundle_alignment."
"optional_aiguard_source_traceability must be an object when present",
)
if not isinstance(traceability, dict):
return

_record(
traceability.get("context_role")
== OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_CONTEXT_ROLE,
errors,
"EdgeEnv handoff lab_bundle_alignment."
"optional_aiguard_source_traceability.context_role must be "
f"{OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_CONTEXT_ROLE}",
)
_record(
traceability.get("edgeenv_does_not_generate_guard_analysis") is True,
errors,
"EdgeEnv handoff lab_bundle_alignment."
"optional_aiguard_source_traceability."
"edgeenv_does_not_generate_guard_analysis must be true",
)
_record(
traceability.get("lab_is_final_decision_owner") is True,
errors,
"EdgeEnv handoff lab_bundle_alignment."
"optional_aiguard_source_traceability.lab_is_final_decision_owner "
"must be true",
)
_record(
traceability.get("optional_present_source_artifact")
== OPTIONAL_AIGUARD_SOURCE_ARTIFACT,
errors,
"EdgeEnv handoff lab_bundle_alignment."
"optional_aiguard_source_traceability.optional_present_source_artifact "
"must match the Lab-known AIGuard optional stale-drop source artifact",
)


def _has_optional_aiguard_evidence_declaration(
handoff: dict[str, Any],
) -> bool:
Expand All @@ -1900,6 +1989,25 @@ def _has_optional_aiguard_evidence_declaration(
)


def _has_optional_aiguard_source_traceability_declaration(
handoff: dict[str, Any],
) -> bool:
alignment = handoff.get("lab_bundle_alignment")
if not isinstance(alignment, dict):
return False
traceability = alignment.get("optional_aiguard_source_traceability")
if not isinstance(traceability, dict):
return False
return (
traceability.get("context_role")
== OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_CONTEXT_ROLE
and traceability.get("edgeenv_does_not_generate_guard_analysis") is True
and traceability.get("lab_is_final_decision_owner") is True
and traceability.get("optional_present_source_artifact")
== OPTIONAL_AIGUARD_SOURCE_ARTIFACT
)


def _validate_remote_runtime_event_summary_evidence(
item: dict[str, Any],
index: int,
Expand Down Expand Up @@ -3132,6 +3240,7 @@ def _write_summary(
errors: list[str],
edgeenv_handoff_present: bool,
optional_aiguard_evidence_declared: bool,
optional_aiguard_source_traceability_declared: bool,
) -> None:
if not path:
return
Expand All @@ -3140,6 +3249,10 @@ def _write_summary(
contract_markers.extend(EDGEENV_HANDOFF_SUMMARY_CONTRACT_MARKERS)
if optional_aiguard_evidence_declared:
contract_markers.append(EDGEENV_HANDOFF_OPTIONAL_AIGUARD_SUMMARY_MARKER)
if optional_aiguard_source_traceability_declared:
contract_markers.append(
EDGEENV_HANDOFF_OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_SUMMARY_MARKER
)
lines = [
"# Runtime Intelligence Bundle Manifest Gate",
"",
Expand Down Expand Up @@ -3173,12 +3286,16 @@ def main(
_validate_manifest_shape(manifest_payload, errors)
handoff_payload: dict[str, Any] | None = None
optional_aiguard_evidence_declared = False
optional_aiguard_source_traceability_declared = False
if edgeenv_handoff:
edgeenv_handoff_path = Path(edgeenv_handoff).resolve()
handoff_payload = _load_json(edgeenv_handoff_path, "EdgeEnv handoff manifest")
optional_aiguard_evidence_declared = (
_has_optional_aiguard_evidence_declaration(handoff_payload)
)
optional_aiguard_source_traceability_declared = (
_has_optional_aiguard_source_traceability_declaration(handoff_payload)
)
_validate_edgeenv_handoff_alignment(
handoff_payload,
handoff_path=edgeenv_handoff_path,
Expand Down Expand Up @@ -3229,6 +3346,9 @@ def main(
errors=errors,
edgeenv_handoff_present=bool(edgeenv_handoff),
optional_aiguard_evidence_declared=optional_aiguard_evidence_declared,
optional_aiguard_source_traceability_declared=(
optional_aiguard_source_traceability_declared
),
)
if errors:
rprint("[red]Runtime Intelligence bundle manifest gate failed.[/red]")
Expand Down
1 change: 1 addition & 0 deletions scripts/check_runtime_intelligence_ci_artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@
"edgeenv_handoff: orchestrator_task_event_rollup validated",
"edgeenv_handoff: missing_telemetry_orchestrator_context validated",
"edgeenv_handoff: optional AIGuard evidence types declared",
"edgeenv_handoff: optional AIGuard source traceability declared",
)
REQUIRED_LAB_EXPECTED_REPORT_MARKERS = (
"Runtime Intelligence Risk Summary",
Expand Down
40 changes: 40 additions & 0 deletions tests/test_runtime_intelligence_bundle_manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
from pathlib import Path

from scripts.check_runtime_intelligence_bundle_manifest import (
OPTIONAL_AIGUARD_SOURCE_ARTIFACT,
OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_CONTEXT_ROLE,
REQUIRED_EXPECTED_REPORT_MARKERS,
main as manifest_gate,
)
Expand Down Expand Up @@ -163,6 +165,8 @@ def test_runtime_intelligence_docs_describe_expected_report_markers():
for doc in (handoff_doc, ci_doc):
assert "expected_report_markers" in doc
assert "optional_aiguard_evidence_types" in doc
assert "optional_aiguard_source_traceability" in doc
assert "read_only_optional_source_traceability" in doc
assert "edgeenv_orchestrator_stale_drop_summary" in doc
assert "Lab-owned Runtime Intelligence report contract" in doc
for marker in REQUIRED_EXPECTED_REPORT_MARKERS:
Expand Down Expand Up @@ -327,6 +331,13 @@ def test_runtime_intelligence_bundle_manifest_gate_validates_edgeenv_handoff(
== 0
)
summary = summary_path.read_text(encoding="utf-8")
handoff = json.loads(EDGEENV_HANDOFF.read_text(encoding="utf-8"))
assert handoff["lab_bundle_alignment"]["optional_aiguard_source_traceability"] == {
"context_role": OPTIONAL_AIGUARD_SOURCE_TRACEABILITY_CONTEXT_ROLE,
"edgeenv_does_not_generate_guard_analysis": True,
"lab_is_final_decision_owner": True,
"optional_present_source_artifact": OPTIONAL_AIGUARD_SOURCE_ARTIFACT,
}
assert "edgeenv_handoff: lab_bundle_alignment validated" in summary
assert "edgeenv_handoff: runtime_telemetry_history validated" in summary
assert "edgeenv_handoff: history_seed_run_config validated" in summary
Expand All @@ -346,6 +357,10 @@ def test_runtime_intelligence_bundle_manifest_gate_validates_edgeenv_handoff(
in summary
)
assert "edgeenv_handoff: optional AIGuard evidence types declared" in summary
assert (
"edgeenv_handoff: optional AIGuard source traceability declared"
in summary
)


def test_runtime_intelligence_bundle_manifest_gate_fails_for_bad_seed_run_config(
Expand Down Expand Up @@ -670,6 +685,31 @@ def test_runtime_intelligence_bundle_manifest_gate_fails_for_bad_optional_guard_
) in summary


def test_runtime_intelligence_bundle_manifest_gate_fails_for_bad_optional_source_traceability(
tmp_path,
):
handoff = json.loads(EDGEENV_HANDOFF.read_text(encoding="utf-8"))
handoff["lab_bundle_alignment"]["optional_aiguard_source_traceability"][
"optional_present_source_artifact"
]["repository"] = "InferEdgeEnv"
handoff_path = tmp_path / "edgeenv_lab_handoff_manifest.json"
handoff_path.write_text(json.dumps(handoff), encoding="utf-8")
summary_path = tmp_path / "bundle_manifest_gate_summary.md"

result = manifest_gate(
manifest=str(MANIFEST),
edgeenv_handoff=str(handoff_path),
summary_out=str(summary_path),
)

assert result == 2
summary = summary_path.read_text(encoding="utf-8")
assert (
"optional_aiguard_source_traceability.optional_present_source_artifact "
"must match the Lab-known AIGuard optional stale-drop source artifact"
) in summary


def test_runtime_intelligence_bundle_manifest_gate_fails_for_missing_report_marker(
tmp_path,
):
Expand Down
3 changes: 3 additions & 0 deletions tests/test_runtime_intelligence_ci_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,7 @@ def test_runtime_intelligence_ci_artifact_gate_passes_for_expected_outputs(tmp_p
"- edgeenv_handoff: remote_dispatch_boundary preserved",
"- edgeenv_handoff: external AIGuard evidence requirements declared",
"- edgeenv_handoff: optional AIGuard evidence types declared",
"- edgeenv_handoff: optional AIGuard source traceability declared",
"- edgeenv_handoff: device_local_producer_lineage validated",
"- edgeenv_handoff: fixture_matrix_context validated",
"- edgeenv_handoff: producer_lineage_guard_alignment validated",
Expand Down Expand Up @@ -668,6 +669,7 @@ def test_runtime_intelligence_ci_artifact_gate_fails_for_missing_lab_marker_cont
"- edgeenv_handoff: remote_dispatch_boundary preserved",
"- edgeenv_handoff: external AIGuard evidence requirements declared",
"- edgeenv_handoff: optional AIGuard evidence types declared",
"- edgeenv_handoff: optional AIGuard source traceability declared",
"- edgeenv_handoff: device_local_producer_lineage validated",
"- edgeenv_handoff: fixture_matrix_context validated",
"- edgeenv_handoff: producer_lineage_guard_alignment validated",
Expand Down Expand Up @@ -1034,6 +1036,7 @@ def test_runtime_intelligence_ci_artifact_gate_fails_for_failed_deployment_risk(
"- edgeenv_handoff: remote_dispatch_boundary preserved",
"- edgeenv_handoff: external AIGuard evidence requirements declared",
"- edgeenv_handoff: optional AIGuard evidence types declared",
"- edgeenv_handoff: optional AIGuard source traceability declared",
"- edgeenv_handoff: device_local_producer_lineage validated",
"- edgeenv_handoff: fixture_matrix_context validated",
"- edgeenv_handoff: producer_lineage_guard_alignment validated",
Expand Down
4 changes: 4 additions & 0 deletions tests/test_runtime_intelligence_smoke_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@ def test_runtime_intelligence_smoke_script_runs_artifact_chain(tmp_path):
in bundle_summary
)
assert "edgeenv_handoff: optional AIGuard evidence types declared" in bundle_summary
assert (
"edgeenv_handoff: optional AIGuard source traceability declared"
in bundle_summary
)
assert "edgeenv_handoff: device_local_producer_lineage validated" in bundle_summary
assert (
"edgeenv_handoff: producer_lineage_guard_alignment validated"
Expand Down
Loading