Skip to content

[codex] Add golden AL distributions#1926

Open
functionstackx wants to merge 2 commits into
mainfrom
codex/add-golden-al-distributions
Open

[codex] Add golden AL distributions#1926
functionstackx wants to merge 2 commits into
mainfrom
codex/add-golden-al-distributions

Conversation

@functionstackx

@functionstackx functionstackx commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • build on @qiching 's work & @haic0 's work on AL collector
  • add golden acceptance-length distributions for DeepSeek V4 Pro and Qwen3.5 MTP
  • add golden acceptance-length distributions for Kimi K2.5 and MiniMax-M3 EAGLE3
  • preserve collection metadata and link every YAML to its source Actions run
  • document the SPEED-Bench collection methodology and AgentX synthetic-acceptance policy in English and Simplified Chinese

Golden-value sources

Methodology sources

Validation

  • confirmed every committed AL value and metadata block against its run's speedbench-reference-al artifact
  • parsed all four golden files with PyYAML
  • verified every relative README link resolves and both language versions contain matching link targets and section counts
  • verified balanced Markdown code fences
  • ran git diff --check

@functionstackx functionstackx marked this pull request as ready for review June 24, 2026 22:25
@functionstackx functionstackx requested a review from a team June 24, 2026 22:25
@claude

claude Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Claude finished @functionstackx's task in 52s —— View job


Review: PR #1926 — Add golden AL distributions

  • Read changed files (4 YAML + 2 README)
  • Check YAML validity & metadata consistency
  • Verify README links and cross-language parity
  • Post review

LGTM — no blocking issues found.

This is a docs/data-only change (4 golden AL YAMLs + EN/ZH READMEs), no executable code paths. Spot checks all passed:

  • Every relative link resolves: .github/workflows/speedbench-al.yml and all four referenced collectors under benchmarks/single_node/speedbench/ (dsv4_fp4_b300_vllm.sh, qwen3.5_fp4_b300_vllm.sh, kimik2.5_fp4_b300_vllm.sh, minimaxm3_fp4_b300_vllm.sh) exist on this branch.
  • The README example value synthetic_acceptance_length: 3.24 is internally consistent — it matches kimik2.5_eagle3.yamlthinking_on → draft length 4.
  • All four YAMLs are well-formed (integer draft-length keys 1–8, float AL values, per-model thinking_on/thinking_off blocks) and each carries its source Actions-run URL on line 1 plus full sampling metadata.
  • EN and ZH READMEs are structurally parallel (matching section count, table rows, and link targets).
  • The small non-monotonic dips at higher draft lengths (e.g. dsv4 thinking_on 6→7: 2.55→2.54; qwen3.5 thinking_off 7→8: 4.73→4.71) are expected for real measured AL and not data errors.
    · codex/add-golden-al-distributions

@functionstackx functionstackx force-pushed the codex/add-golden-al-distributions branch from c1c98b9 to fb9a9ca Compare June 24, 2026 22:25
@functionstackx functionstackx requested review from andyluo7, chunfangamd, haic0, qiching and xinli-sw and removed request for andyluo7 June 24, 2026 22:26
@functionstackx

functionstackx commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator Author

hi @chunfangamd @haic0 @xinli-sw @qiching PTAL before i merge this PR tmw at 4pm ET

Comment thread golden_al_distribution/README.md Outdated
Co-authored-by: functionstackx <functionstackx@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants