feat(report): record run_config (load/runtime settings) in result_summary.json

[nvzhihanj]

Summary

Record the run's load/runtime configuration in the benchmark report so a run is self-describing and reproducible from its own result_summary.json.

A single run_config field on Report is populated from the Pydantic config in one line:

settings.model_dump(mode="json", include={"runtime", "load_pattern", "warmup"})

The config model is the single source of truth — new/renamed config fields propagate automatically, with no hand-listing at the from_snapshot/finalize call site. Excluded: client (transport plumbing, not run-defining) and endpoint_config (api_key / endpoint URLs — a sibling of settings, so no secrets leak).

• Serialized into result_summary.json, and rendered as a Run config: block in report.txt.
• Replaces the standalone seeds field (Feat: Add required fields (TPS/QPS/seed) into results_summary.json #353): the RNG seeds now live under run_config.runtime / run_config.warmup.

Example result_summary.json

From a real DGX-B300x8 DeepSeek-R1 server (poisson) run — qps/tps stay top-level, the run's settings (incl. seeds) are under run_config (metric dicts elided as { ... }):

{
  "version": "0.1.0",
  "git_sha": "...",
  "n_samples_completed": 40000,
  "duration_ns": 3328465383747,
  "state": "complete",
  "complete": true,
  "ttft": { ... }, "tpot": { ... }, "latency": { ... }, "output_sequence_lengths": { ... },
  "legacy_loadgen_window_duration_ns": 2753346633148,
  "qps": 14.53,
  "tps": 54489.98,
  "run_config":
  {
    "runtime": {
      "min_duration_ms": 600000,
      "max_duration_ms": 3600000,
      "n_samples_to_issue": 40000,
      "scheduler_random_seed": 42,
      "dataloader_random_seed": 42
    },
    "load_pattern": {
      "type": "poisson",
      "target_qps": 14.75,
      "target_concurrency": null,
      "use_legacy_loadgen_qps_metrics": true
    },
    "warmup": {
      "enabled": false,
      "n_requests": null,
      "salt": true,
      "drain": false,
      "warmup_random_seed": 42
    }
  }
}

Same, rendered in report.txt:

Run config:
  runtime: n_samples_to_issue=40000, scheduler_random_seed=42, dataloader_random_seed=42, min_duration_ms=600000, max_duration_ms=3600000
  load_pattern: type=poisson, target_qps=14.75, target_concurrency=None, use_legacy_loadgen_qps_metrics=True
  warmup: enabled=False, salt=True, drain=False, warmup_random_seed=42

Tests

• run_config keyword-only passthrough; serialization + report.txt rendering; absent -> null.

Rebased on latest main (keeps PR #372's use_legacy_loadgen_qps_metrics; git_sha handled by #398).

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[gemini-code-assist]

Code Review

This pull request introduces a mechanism to resolve the package's git commit SHA using git archive's export-subst feature via a new _git_archival.txt file, falling back to running git rev-parse anchored to the package directory. It also reorders the serialized JSON output of metrics reports to surface key throughput and run identity fields first. Feedback is provided regarding a potential issue where git rev-parse could traverse up to a parent repository if the package is installed in a virtual environment within an unrelated git repository, along with a suggestion to check for the existence of the .git directory first.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[arekay-nv]

Should we replicate sections of the config.yaml here - if the goal is reproducibility, we will still need the config.yaml file to run unless we include the whole thing. Otherwise we might end up with inconsistency between the two.

[viraatc]

The saved run_config can differ from what actually ran.

Audit phases override the sample count and ordering in ctx.rt_settings. For example, TEST04 runs one phase with normal sampling and another that repeatedly issues one fixed sample. This code instead saves ctx.config.settings, which does not contain those overrides. Consequently, both phases produce identical run_config blocks and report n_samples_to_issue: null.

Suggested fix: record effective runtime settings from ctx.rt_settings, including the resolved sample count from total_samples_to_issue(), sample_order.kind, and sample_order.fixed_index. Consider placing these under run_config["resolved_runtime"] so the original user configuration remains distinct from resolved values. Please add an integration assertion that TEST04 reference and output-caching phase configurations differ.