Skip to content

Add audeering wav2vec2 dimensional emotion (speech regression) support#1084

Open
DingmaomaoBJTU wants to merge 3 commits into
mainfrom
add-audeering-wav2vec2-emotion
Open

Add audeering wav2vec2 dimensional emotion (speech regression) support#1084
DingmaomaoBJTU wants to merge 3 commits into
mainfrom
add-audeering-wav2vec2-emotion

Conversation

@DingmaomaoBJTU

@DingmaomaoBJTU DingmaomaoBJTU commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

This PR adds support for audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim, routed as audio-classification with an emotion regression head (wav2vec2_emotion_regression). The shipped outcome is L1 on the CPU target EP, and the produced ONNX artifact is fp32 with a matching fp32 recipe name. The highest Goal verdict honestly reached by the tester is L2 PASS, with cosine 0.9999998807907104 and max_abs 1.6689300537109375e-06.

  1. Recipe path(s)

    • examples/recipes/audeering_wav2vec2-large-robust-12-ft-emotion-msp-dim/audio-classification_fp32_config.json
  2. README row

    • true
  3. Build output dir

    • temp\fix_build\
  4. Build log

    • ✅ Build complete in 256.5s
  5. Appended findings

    • model_knowledge/wav2vec2.json findings wav2vec2-001..wav2vec2-005 were appended on the skill repo Lane A, not in this model PR.
    • Captured: custom-head requirement + routing; L2 numeric parity cosine 0.9999998807907104 / max-abs 1.6689e-6; fp32 artifact; op-coverage 410 ops/14 unique; CPU fp32 perf representative median avg 381.46 ms across four runs (range 377.25-401.01 ms), throughput 2.49-2.65 samples/sec, RAM delta +690.7 to +697.0 MB.
    • Model ~165,336,707 params; trace coverage 71/231 modules; autoconf filled optim.gelu_fusion=true, optim.matmul_add_fusion=true, quant stayed null.
  6. Optimum-coverage probe

    • VENDOR-ONLY
    • added_by_winml=[]
  7. Claimed (Effort, Goal, Outcome)

    • Effort: L1
    • Goal ceiling: L2
    • Outcome: L1
    • Target EPs: ["cpu"]
    • Catalog gate: baseline_build=FAIL, verdict=real-engineering, origin_main_commit=130acfe42523b8aa553b1dd10eecd7a1328b832e
    • Baseline command (no recipe):
winml build -m audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim -o temp\review_baseline_fp32\
  • Baseline HEAD: 130acfe42523b8aa553b1dd10eecd7a1328b832e (current origin/main at rebase time)
  • winml --version: winml, version 0.2.0
  • Baseline fail output:
[WinML] Installing Execution Provider: QNNExecutionProvider

Downloading... ░░░░░░░░░░   0%
Downloading... ██████████ 100%
Downloading... ██████████ 100%
QNNExecutionProvider EP installed successfully.
- Version: 2.2451.48.0
- Package Family Name: MicrosoftCorporationII.WinML.Qualcomm.QNN.EP.2_8wekyb3d8bbwe
Usage: winml build [OPTIONS]
Try 'winml build --help' for help.

Error: wav2vec2 doesn't support task text-classification for the onnx backend. Supported tasks are: feature-extraction, automatic-speech-recognition, audio-classification, audio-frame-classification, audio-xvector.
  • Recipe-build delta: the same model succeeds with the checked-in recipe command winml build -c examples\recipes\audeering_wav2vec2-large-robust-12-ft-emotion-msp-dim\audio-classification_fp32_config.json -m audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim -o temp\fix_build\; build exit 0; ✅ Build complete in 256.5s; ONNX input input_values[1,16000], outputs hidden_states[1,1024] and logits[1,3]; initializer dtypes {FLOAT:233, INT64:10} with FLOAT16_COUNT 0, correctly fp32.
  1. Goal-ladder verdict table

    Tier Verdict Command Evidence
    L0 PASS winml build -c examples\recipes\audeering_wav2vec2-large-robust-12-ft-emotion-msp-dim\audio-classification_fp32_config.json -m audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim -o temp\fix_build\ Build exit 0; ✅ Build complete in 256.5s; artifact temp\fix_build\model.onnx; IN [('input_values',[1,16000])]; OUT [('hidden_states',[1,1024]),('logits',[1,3])]; initializer dtypes {FLOAT:233, INT64:10}; FLOAT16_COUNT 0; actual precision fp32, matching recipe filename.
    L1 PASS winml perf -m temp\fix_build\model.onnx --device cpu --ep cpu Four CPU fp32 runs: Avg ms [377.25, 380.48, 382.43, 401.01], representative median avg 381.46 ms; P50 range 347.27-375.69 ms; throughput 2.49-2.65 samples/sec; RAM total delta +690.7 to +697.0 MB. Run-to-run variance includes a run-4 max-latency outlier (1278.88 ms), so the table reports median representative plus range rather than a single best run.
    L2 PASS python temp\audeering_l2_parity.py anchor_onnx_logits [[0.5460754632949829,0.60622638463974,0.4043162763118744]]; anchor_reference_logits [[0.5460754036903381,0.6062266230583191,0.40431657433509827]]; anchor_max_abs 2.980232238769531e-07; random_seed 0; onnx_logits [[0.6837583780288696,0.649890661239624,0.5071567296981812]]; pytorch_logits [[0.6837599873542786,0.6498923301696777,0.507158100605011]]; cosine 0.9999998807907104; max_abs 1.6689300537109375e-06
  2. Methodology-evolution declaration

    • _meta-056 effort-mis-estimate (planner Optimum-probe edit), _meta-057 goal-ceiling mis-estimate (planner Goal-axis edit), and _meta-058 doc-code-drift user_skilldev_skill (explainer/reviewer path fix) were filed on the SKILL repo Lane A branch.
    • The paired skill edits were committed on the skill worktree at commit 9d380bae and are intentionally not included in this model PR to keep lanes unpolluted.
  3. Perf & eval data

EP / Device Precision Verdict Mean p50 Throughput RAM Δ Eval
CPUExecutionProvider / cpu fp32 PASS 381.46 ms representative median avg (runs 377.25, 380.48, 382.43, 401.01 ms) 349.58 ms representative median p50 (run range 347.27-375.69 ms) 2.62 samples/sec representative (range 2.49-2.65) +693.8 MB representative total delta (range +690.7 to +697.0 MB) N/A — ceiling is L2
  1. Component / op-level data
  • Command: winml analyze --model temp\fix_build\model.onnx --ep all --format json
  • Rules parquet count: 1746
  • Command exit code: 1
  • Total operators: 410
  • Unique operator types: 14
  • Per-EP classification: QNNExecutionProvider(NPU): 410/0/0/0 runtime_support=True has_errors=False has_warnings=False. OpenVINOExecutionProvider(NPU): 409/1/0/0 runtime_support=False has_warnings=True partial: OP/ai.onnx/Slice. VitisAIExecutionProvider(NPU): 0/0/0/410 unknown. CPUExecutionProvider: no rule data (all-unknown, expected).
  • Artifact: temp\fix_analyze_output.txt
  1. Reproducible commands
Set-Location C:\Users\qiowu\source\repos\winml-cli
winml build -m audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim -o temp\review_baseline_fp32\
winml build -c examples\recipes\audeering_wav2vec2-large-robust-12-ft-emotion-msp-dim\audio-classification_fp32_config.json -m audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim -o temp\fix_build\
winml perf -m temp\fix_build\model.onnx --device cpu --ep cpu
gh release download v0.2.0 --repo microsoft/winml-cli --pattern "rules-v*.zip" --dir temp; Expand-Archive .\temp\rules-v*.zip -DestinationPath src\winml\modelkit\analyze\rules\runtime_check_rules -Force
winml analyze --model temp\fix_build\model.onnx --ep all --format json
python temp\audeering_l2_parity.py

@DingmaomaoBJTU

Copy link
Copy Markdown
Collaborator Author

REVIEWER verdict: REQUEST_CHANGES

I independently re-ran the reviewer checklist from PR head da4d514a8d4238394a74ad028d2a4c7d8613016c. This is fixable, but I cannot approve from current evidence.

Blocking items (producer action required)

  1. Baseline freshness (_meta-052 / reviewer.md L100) — PR cites baseline origin_main_commit=670ad35e..., but after git fetch origin main, git rev-list --count 670ad35e..origin/main returned 1; current origin/main is 130acfe4. Fix: rebase onto current origin/main and re-run the no--c baseline; update PR item 7/body with the new baseline HEAD.
  2. Baseline-build gate evidence incomplete (reviewer.md L99) — PR body item 7 names baseline_build=FAIL but does not quote the required no--c baseline command output plus winml --version and git rev-parse HEAD. Reviewer re-run: winml --version0.2.0; winml build -m audeering/... -o temp\review_baseline\Error: wav2vec2 doesn't support task text-classification ..., no ✅ Build complete. Fix: after rebase, paste the exact baseline command, fail output, version, and baseline HEAD into the PR body.
  3. Goal-L0 precision mismatch (_meta-014 / reviewer.md L110) — recipe path is audio-classification_fp16_config.json, but recipe has quant: null (...\audio-classification_fp16_config.json:36) and reviewer initializer scan of temp\review_build\model.onnx returned DTYPES {1:233, 7:10} / FLOAT16_COUNT 0; winml perf also reports Model Precision: fp32. Fix: either rename the recipe to _fp32_config.json and update README/PR body references, or add a real fp16/quantization path that emits FLOAT16 initializers.
  4. L1 perf number not independently verified (reviewer.md L111, fail-closed) — PR body claims CPU avg 161.38 ms, but reviewer re-run on the PR-built artifact returned Avg 416.38 ms, Throughput 2.40 samples/sec, RAM +697.4 MB. Fix: after rebase/precision fix, re-run winml perf -m <fresh artifact> --device cpu --ep cpu, paste the fresh log and update item 10 to match the actual run (or explain host variance with evidence).

Checked evidence

  • PR is real/draft: gh pr view 1084 returned draft PR head da4d514a....
  • Checkout/diff scope: git rev-parse HEAD = da4d514a...; git diff --name-only origin/main...HEAD returned exactly examples/recipes/README.md, the audeering recipe JSON, src/winml/modelkit/models/hf/__init__.py, src/winml/modelkit/models/hf/wav2vec2.py.
  • Recipe sanity: input input_values float32 [1,16000], outputs hidden_states/logits, variant loader wav2vec2_emotion_regression (audio-classification_fp16_config.json:12-42).
  • Build: winml build -c ... -m audeering/... -o temp\review_build\ produced ✅ Build complete in 263.8s.
  • Structure: ONNX IR 8, opset 17, inputs [('input_values',[1,16000])], outputs [('hidden_states',[1,1024]),('logits',[1,3])].
  • External data: temp\review_build contains model.onnx plus colocated model.onnx.data (661,342,208 bytes).
  • L2 parity: reviewer script temp\review_l2_parity.py passed; zeros ONNX logits [[0.54607546,0.60622638,0.40431628]], reference max abs 2.98e-7; random seed 0 cosine 1.0, max abs 8.94e-7.
  • Analyze: recursive parquet count 1746; winml analyze --ep all produced 410 ops / 14 unique; QNN 410/0/0/0, OpenVINO 409/1/0/0 partial Slice, VitisAI 0/0/0/410 unknown.
  • Code/design: custom code is confined to per-arch src/winml/modelkit/models/hf/wav2vec2.py; @register_onnx_overwrite at wav2vec2.py:73; imported in __init__.py:98-103; mapping included at __init__.py:119-138; routing path threads loader.model_type through commands/build.py:1602-1608, loader/hf.py:227-237, and loader/resolution.py:452-453,515-517. No new shared hardcoded if model_type == branch was found.
  • Tests: no consistent per-model pytest convention for all neighbouring models/hf additions was found (e.g. recent ViTPose added code without adjacent tests); existing relevant loader scope passed: uv run pytest tests\integration\loader\test_hf_model_class_mapping.py tests\integration\loader\test_load_hf_model.py -q24 passed.
  • Knowledge capture: Lane-A model_knowledge\wav2vec2.json has populated scope.validated_on (lines 17-19, 43-45, etc.), analyze totals/op types (lines 89-91), export metadata 165,336,707 params and 71/231 trace (lines 15, 41, 120), and autoconf diff (lines 15, 41, 66).
  • Optimum probe: exact reviewer probe with import optimum.exporters.onnx.model_configs returned vendor tasks ['audio-classification', 'audio-frame-classification', 'audio-xvector', 'automatic-speech-recognition', 'feature-extraction'], after_winml same, added_by_winml=[]; L1-with-code remains justified by checkpoint-specific head evidence in _meta-056 and baseline failure.
  • Methodology declaration: PR item 9 declares _meta-056/_meta-057/_meta-058 and says skill edits are Lane A; per this review handoff, those edits are audited in the skill repo, not required in this Lane-B PR.

Reachable verified EP: CPUExecutionProvider/cpu. Deferred target EPs: none (target_eps is ["cpu"]).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@DingmaomaoBJTU DingmaomaoBJTU force-pushed the add-audeering-wav2vec2-emotion branch from da4d514 to f3b531a Compare July 10, 2026 05:11
@DingmaomaoBJTU

Copy link
Copy Markdown
Collaborator Author

Addressed all 4 items in f3b531a: rebased onto origin/main 130acfe, renamed recipe fp16→fp32 (0 FLOAT16 initializers confirmed), added baseline-FAIL gate evidence to item 7, re-ran CPU perf (representative median avg 381.46 ms across runs 377.25, 380.48, 382.43, 401.01 ms; variance noted). Ready for re-review.

@DingmaomaoBJTU

Copy link
Copy Markdown
Collaborator Author

APPROVE

Verified PR #1084 from fresh PR head f3b531a6ab1237af1a4135ec6f1c3b2b175b9c84 on branch add-audeering-wav2vec2-emotion.

Checkout / scope

  • Branch/head: git rev-parse --abbrev-ref HEAD => add-audeering-wav2vec2-emotion; git rev-parse HEAD => f3b531a6ab1237af1a4135ec6f1c3b2b175b9c84.
  • Rebase/base freshness: git merge-base HEAD origin/main => 130acfe42523b8aa553b1dd10eecd7a1328b832e, equal to git rev-parse origin/main; HEAD is 0 commits behind current main.
  • GitHub mergeability: gh pr view 1084 --json headRefOid,headRefName,mergeable,isDraft => head f3b531a6..., mergeable: MERGEABLE, draft PR.
  • Lane-B diff scope (git diff --name-only origin/main...HEAD) is exactly:
    • examples/recipes/README.md
    • examples/recipes/audeering_wav2vec2-large-robust-12-ft-emotion-msp-dim/audio-classification_fp32_config.json
    • src/winml/modelkit/models/hf/__init__.py
    • src/winml/modelkit/models/hf/wav2vec2.py
  • No copilot-skills/, SKILL.md, agents/*.md, tests/, or unrelated src/ churn in the PR diff.

Engineering/code review

  • src/winml/modelkit/models/hf/wav2vec2.py:28-63 transcribes the model-card RegressionHead + EmotionModel exactly: wav2vec2 backbone, mean pooling over sequence, dense/tanh/dropout/out_proj regression head, outputs (hidden_states, logits).
  • src/winml/modelkit/models/hf/wav2vec2.py:73-93 registers an ONNX config for wav2vec2_emotion_regression / audio-classification with raw input_values dynamic axes and hidden_states + logits batch axes; this matches the recipe and emitted ONNX.
  • src/winml/modelkit/models/hf/wav2vec2.py:96-104 uses the established custom-wrapper pattern: hyphenated lookup key in MODEL_CLASS_MAPPING, plus register_specialization(..., WinMLModelForGenericTask).
  • src/winml/modelkit/models/hf/__init__.py:98-103,119-138 imports the new module and merges its mapping, so decorators and class lookup run at import time.
  • examples/recipes/.../audio-classification_fp32_config.json:12-42 declares input_values float32 [1,16000], value_range [-1,1], outputs hidden_states/logits, quant: null, and loader override model_type=wav2vec2_emotion_regression, model_class=EmotionModel. No hardcoded model branching was added in shared paths (rg 'if model_type ==' src\winml\modelkit found only pre-existing utility/catalog entries).

PR body / report audit

  • The PR body contains all 12 Step-6 sections: recipe path, README row, build dir/log, appended findings, Optimum probe, claimed tiers + baseline gate, Goal ladder, methodology declaration, perf/eval table, component/analyze data, reproducible commands (temp\pr1084_body_recheck.md:3-93).
  • Baseline-gate evidence is present in body item 7 with the no-recipe command and exact error string (temp\pr1084_body_recheck.md:29-53).
  • Methodology-evolution declaration is present (temp\pr1084_body_recheck.md:65-67); skill changes are explicitly kept out of this Lane-B model PR, consistent with the observed 4-file Lane-B scope.
  • Perf/eval item 10 is per target EP/device with CPU runtime row only; target EPs are ['cpu'], so no DML/QNN/OpenVINO runtime PASS is fabricated (temp\pr1084_body_recheck.md:69-73). Static analyze notes for other EPs are in the component section, not claimed as runtime L1.
  • The body no longer has 161 as a headline or claim; it reports representative CPU avg 381.46 ms with run range (Select-String '161' no matches; temp\pr1084_body_recheck.md:17,62,73).

Independent rebuild and artifact verification

  • Rebuilt from PR-head recipe using the body command after deleting stale temp\fix_build: winml build -c examples\recipes\audeering_wav2vec2-large-robust-12-ft-emotion-msp-dim\audio-classification_fp32_config.json -m audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim -o temp\fix_build\.
  • Build result: exit 0, stdout contains ✅ Build complete in 240.1s; log saved at temp\review1084_build.log; final artifact temp\fix_build\model.onnx.
  • L0 structural ONNX check: IR 8, opset 17; input input_values float32 [1,16000]; outputs hidden_states float32 [1,1024], logits float32 [1,3].
  • Precision honesty: initializer dtype counts {FLOAT: 233, INT64: 10}, FLOAT16_COUNT=0; recipe filename is now _fp32_ and winml_build_config.json has quant: null.
  • External-data layout is valid: model.onnx and model.onnx.data are both in temp\fix_build\.
  • Build artifacts read directly: winml_build_config.json loader is {task: audio-classification, model_class: EmotionModel, model_type: wav2vec2_emotion_regression}, optim autoconf filled gelu_fusion=true, matmul_add_fusion=true, quant null; export_htp_metadata.json class EmotionModel, 165,336,707 parameters, trace 71/231; analyze_result.json has total_operators=410, unique_operator_types=14, top op counts include Reshape=122, Gemm=75, Transpose=63, and QNN static-rule row is supported.

Goal ladder re-verification

  • L0 PASS: build + structural check above.
  • L1 PASS on target CPU: three independent winml perf -m temp\fix_build\model.onnx --device cpu --ep cpu runs yielded avg latencies 376.84 ms, 381.93 ms, 433.75 ms; throughputs 2.65, 2.62, 2.31 samples/sec; total RAM deltas +696.1 MB, +696.5 MB, +688.8 MB. Logs: temp\review1084_perf1.log..temp\review1084_perf3.log. These are in honest variance of the PR's representative 381.46 ms / 2.62 samples/sec; no threshold gate applies.
  • L2 PASS: direct ONNX Runtime CPU inference on zeros float32[1,16000] produced logits [[0.5460754633, 0.6062263846, 0.4043162763]]; reference is [[0.5460754, 0.6062266, 0.40431657]]; max-abs 2.98e-07 (<1e-4). This proves the trained regression head exported, not a random stock head.
  • Optimum coverage probe re-run for native wav2vec2: vendor and after-winml tasks both ['audio-classification','audio-frame-classification','audio-xvector','automatic-speech-recognition','feature-extraction'], added_by_winml=[]; the L1 code remains justified by this model's non-loadable custom regression head.

Baseline gate

  • Re-ran no-recipe baseline: winml build -m audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim -o temp\review_baseline_fp32\.
  • Result: exit code 2 with Error: wav2vec2 doesn't support task text-classification for the onnx backend. Supported tasks are: feature-extraction, automatic-speech-recognition, audio-classification, audio-frame-classification, audio-xvector. Log: temp\review1084_baseline.log.
  • Therefore the recipe + variant routing are load-bearing and the baseline fail -> recipe success delta is real on current origin/main.

Prior REQUEST_CHANGES items

  1. Stale rebase base: FIXED. Merge-base equals current origin/main (130acfe42523b8aa553b1dd10eecd7a1328b832e), and PR is MERGEABLE.
  2. Baseline-gate evidence missing from PR body: FIXED. Body item 7 includes the failing command and exact error; independent baseline re-run reproduced it.
  3. fp16 filename / fp32 artifact mismatch: FIXED. Diff contains only audio-classification_fp32_config.json; no _fp16_ audeering recipe remains; quant:null; rebuilt artifact has 0 FLOAT16 initializers.
  4. Perf honesty: FIXED. Body no longer headlines 161ms; reports representative 381.46 ms with range. My three-run perf (376.84, 381.93, 433.75 ms) confirms that is honest host-load variance.

EP coverage annotation

  • charter.target_eps = [cpu].
  • Reachable verified: CPUExecutionProvider / cpu completed L0, L1, and L2.
  • Deferred target EPs: none. DML/QNN/OpenVINO/VitisAI are not target runtime EPs for this charter; any NPU/GPU entries are static analyze/rules-only, not runtime PASS.
  • Coverage: full for the only target EP (cpu).

Sign-off

  • Verdict: APPROVE.
  • Verified commit: f3b531a6ab1237af1a4135ec6f1c3b2b175b9c84.
  • Build output dir: C:\Users\qiowu\source\repos\winml-cli\temp\fix_build\.
  • Logs/artifacts produced: temp\review1084_build.log, temp\review1084_perf1.log, temp\review1084_perf2.log, temp\review1084_perf3.log, temp\review1084_baseline.log, temp\pr1084_body_recheck.md.

@DingmaomaoBJTU DingmaomaoBJTU marked this pull request as ready for review July 10, 2026 05:48
@DingmaomaoBJTU DingmaomaoBJTU requested a review from a team as a code owner July 10, 2026 05:48
nn.Linear.__call__ is typed to return Any, so 
eturn x tripped the strict
mypy [no-any-return] gate (CI lint). Return the projection via
cast("torch.Tensor", ...) to match the repo house style (blip.py/mu2.py) and
drop the now-unnecessary RET504 assign-then-return. Runtime behavior unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

@xieofxie xieofxie left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, clean addition — the routing follows house style well: hyphenated MODEL_CLASS_MAPPING keys that rely on the resolver's _- normalization (consistent with the qwen3-* variants), register_specialization(..., "WinMLModelForGenericTask"), and the model_type_override plumbing via the recipe loader block. Lint passes and the RegressionHead/mean-pool forward match the audeering reference, which lines up with the L2 cosine ≈1.0 you reported.

My main ask is test coverage — see the inline note on wav2vec2.py. Every comparable model-class/ONNX-config addition ships a small unit test, and this one doesn't. The rest are minor nits.

}


MODEL_CLASS_MAPPING: dict[tuple[str, str], type] = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No unit test ships with this module. Every comparable MODEL_CLASS_MAPPING / OnnxConfig addition has one under tests/unit/models/ — e.g. test_vitpose_mapping.py, segformer/test_onnx_config.py, sam2/test_onnx_config.py, and the qwen3-* modeling tests. A tiny test here would lock in the behavior that actually matters:

  • ("wav2vec2-emotion-regression", "audio-classification") is present in the aggregated MODEL_CLASS_MAPPING and resolves to EmotionModel;
  • resolve_task(cfg, task="audio-classification", model_class="EmotionModel", model_type_override="wav2vec2_emotion_regression") returns EmotionModel (guards the _- normalization contract this whole file depends on);
  • the OnnxConfig exposes the expected inputs/outputs axes.

Without it, a rename of EMOTION_REGRESSION_MODEL_TYPE or a resolver change to the normalization could silently break routing with nothing failing in CI.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tests/unit/models/wav2vec2/test_onnx_config.py (commit 7110135). It locks in all three behaviors:

  • ("wav2vec2-emotion-regression", "audio-classification") is present in the aggregated MODEL_CLASS_MAPPING and resolves to EmotionModel (plus the module mapping ⊆ aggregate check);
  • resolve_task(cfg, task="audio-classification", model_class="EmotionModel", model_type_override="wav2vec2_emotion_regression") returns EmotionModel, guarding the _- normalization contract;
  • Wav2Vec2EmotionRegressionIOConfig is registered with TasksManager for audio-classification and exposes the expected inputs/outputs axes.

8 tests, all passing.

MODEL_CLASS_MAPPING as _WAV2VEC2_CLASS_MAPPING,
)
from .wav2vec2 import (
Wav2Vec2EmotionRegressionIOConfig as _Wav2Vec2EmotionRegressionIOConfig,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This side-effect import is missing the # triggers registration trailing comment that every other *IOConfig import in this file carries (bart, blip, segformer, qwen3-*, zoedepth, …). Worth adding for consistency so it's clear the import exists for the @register_onnx_overwrite side effect, not for use. It can also be folded into the same from .wav2vec2 import (...) block as the mapping import above rather than a second from .wav2vec2 statement.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the # triggers registration comment and collapsed the mapping import to a single statement (commit 7110135).

I kept it as two from .wav2vec2 statements rather than one block, though: lint.isort.combine-as-imports is false in pyproject.toml, so ruff splits aliased imports back onto separate statements (same reason bart/t5/siglip each get their own line). The comment sits on its own line inside the parens, matching the existing qwen3 long-name imports.

Comment thread examples/recipes/README.md Outdated
## Models

Total: **75** (model, task) tuples that pass fp16 eval on all 10 (EP, device) buckets.
Total: **76** (model, task) tuples with curated model recipes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rewords the shared header from "pass fp16 eval on all 10 (EP, device) buckets" to "with curated model recipes" for all 76 rows. This model is legitimately fp32/CPU-only so it can't meet the old bar — but relaxing the sentence for the whole table quietly drops the fp16/all-buckets guarantee that the other 75 entries do satisfy. Consider keeping the stronger wording and noting the fp32/CPU exception, rather than weakening the guarantee for every row.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by design?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — not by design. Reworded in commit 7110135 to keep the stronger guarantee instead of relaxing it for every row:

Total: 76 (model, task) tuples with curated recipes. 75 pass fp16 eval on all 10 (EP, device) buckets; audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim (audio-classification) is fp32/CPU-only.

So the fp16/all-buckets bar still stands for the other 75 and the fp32/CPU exception is called out explicitly.

Comment thread src/winml/modelkit/models/hf/wav2vec2.py Outdated
- Add tests/unit/models/wav2vec2/test_onnx_config.py covering the
  MODEL_CLASS_MAPPING entry -> EmotionModel, the resolve_task underscore
  (_ -> -) normalization contract, and the IOConfig registration plus
  input/output axes.
- Add the '# triggers registration' comment to the IOConfig side-effect
  import and collapse the mapping import to a single statement.
- Restore the fp16/all-buckets guarantee wording in the recipes README
  and note the new fp32/CPU-only exception instead of weakening every row.
- Drop the unused **kwargs from RegressionHead.forward.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants