Skip to content

Add audio-classification fp16 recipe for audeering/wav2vec2-large-robust-24-ft-age-gender#1094

Open
ssss141414 wants to merge 3 commits into
mainfrom
shzhen/add-wav2vec2-age-gender
Open

Add audio-classification fp16 recipe for audeering/wav2vec2-large-robust-24-ft-age-gender#1094
ssss141414 wants to merge 3 commits into
mainfrom
shzhen/add-wav2vec2-age-gender

Conversation

@ssss141414

Copy link
Copy Markdown
Contributor

Add audio-classification fp16 recipe: audeering/wav2vec2-large-robust-24-ft-age-gender

1. Model & task

audeering/wav2vec2-large-robust-24-ft-age-gender @ audio-classification, fp16.

2. What this PR adds

  • examples/recipes/audeering_wav2vec2-large-robust-24-ft-age-gender/audio-classification_fp16_config.json (new recipe)
  • examples/recipes/README.md (index row + total bump)

Recipe-only. No source code changed (Effort-L0★).

3. Why it's needed (the engineering delta)

This is a custom trust_remote_code model — config.json architectures=["Model"] (an AgeGenderModel: a wav2vec2 backbone with age-regression + gender-classification heads), not a standard Wav2Vec2ForSequenceClassification.

Because of the custom head, winml build auto-config mis-resolves the task to text-classification, and the build fails:

Error: wav2vec2 doesn't support task text-classification for the onnx backend.
Supported tasks are: feature-extraction, automatic-speech-recognition,
audio-classification, audio-frame-classification, audio-xvector.

This recipe pins task=audio-classification (generated via winml config -t audio-classification --trust-remote-code), which makes the model build. winml build has no --task flag, so a checked-in recipe is the only way to override the mis-resolved task.

4. Real-engineering gate (_meta-038)

  • Gate-1 (recipe vs winml config -t audio-classification autoconf): IDENTICAL.
  • Gate-2 (build without recipe): FAILS (auto-config picks text-classification).

Passing Gate-2-fails-without-recipe ⇒ genuine delta, not catalog padding.

5. Build validation (CPU)

Precision Result Time
fp32 exit 0 210.5 s (Export 96.5 + Optimize 45.5)
fp16 exit 0 166.8 s (Export 93.8 + Optimize 41.6 + FP16 29.4)

fp16 artifact confirmed genuinely fp16: 428 FLOAT16 initializers, 0 FLOAT32.

6. Structural (L0)

IR 8, opset 17. IN input_values [1,16000] float32 (1 s @ 16 kHz, value_range [-1,1]). OUT logits [1,3] float32.

7. Perf (L1, CPU / CPUExecutionProvider)

mean 133.95 ms, P50 132.06 ms, P90 140.35 ms, throughput 7.47 samples/sec (20 iters). Model-load +1227 MB RAM, inference +43.7 MB.

8. Scope / honest limits

  • CPU-only. DML / QNN / OpenVINO / VitisAI / NvTensorRTRTX not validated (host availability).
  • No L2 parity vs PyTorch. The eager AgeGenderModel.forward returns an age+gender tuple, not a single logits tensor; the exported audio-classification head is a 3-logit output. A parity harness would need a custom adapter — deferred.
  • No L3 eval (audio-classification default-dataset availability not confirmed).

9. Note on the exported head

The exported graph produces logits[1,3], i.e. the winml audio-classification export path, not the model's native {age scalar, gender 2-way} multi-head. Downstream consumers should expect a 3-logit classification output.

10. Requirements

--trust-remote-code is required at both config and build time (custom modeling code).

11. Testing done

Two builds (fp32 + fp16, both exit 0), fp16 initializer dtype audit, winml perf on CPU. Reproducible from the checked-in recipe via winml build -c <recipe> -m audeering/wav2vec2-large-robust-24-ft-age-gender --trust-remote-code.

12. Follow-ups (out of scope here)

  • Non-CPU EP validation once hosts are available.
  • L2 parity adapter for the custom multi-head model.
  • Skill-side note (separate Lane-A PR): winml build has no --task; the only task override is a pre-generated recipe.

…ust-24-ft-age-gender

Custom trust_remote_code model (AgeGenderModel: wav2vec2 backbone + age/gender heads). Stock 'winml build' auto-config mis-resolves the task to text-classification, which the ONNX backend rejects for wav2vec2. This recipe forces task=audio-classification (via 'winml config -t'), making the model buildable. Validated on CPU: fp32 + fp16 builds succeed (fp16 artifact = 428 FLOAT16 inits / 0 FLOAT32), L1 perf P50 132ms.
@ssss141414 ssss141414 marked this pull request as ready for review July 13, 2026 12:22
@ssss141414 ssss141414 requested a review from a team as a code owner July 13, 2026 12:22
@xieofxie

Copy link
Copy Markdown
Contributor

if without config, could perf run it directly?

@ssss141414

Copy link
Copy Markdown
Contributor Author

if without config, could perf run it directly?

As PR's description, it will failed on build stage without config. And the following command will not be run (perf/ eval)...

Because of the custom head, winml build auto-config mis-resolves the task to text-classification, and the build fails:

Error: wav2vec2 doesn't support task text-classification for the onnx backend.
Supported tasks are: feature-extraction, automatic-speech-recognition,
audio-classification, audio-frame-classification, audio-xvector.

@xieofxie

Copy link
Copy Markdown
Contributor

if without config, could perf run it directly?

As PR's description, it will failed on build stage without config. And the following command will not be run (perf/ eval)...

Because of the custom head, winml build auto-config mis-resolves the task to text-classification, and the build fails:

Error: wav2vec2 doesn't support task text-classification for the onnx backend.
Supported tasks are: feature-extraction, automatic-speech-recognition,
audio-classification, audio-frame-classification, audio-xvector.

same as another pr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants