Skip to content

recipe(wav2vec2): add jakeBland/wav2vec-vm-finetune (audio-classification)#1114

Open
codykk wants to merge 1 commit into
mainfrom
yongyue/add-jakeBland-wav2vec-vm-finetune-recipe
Open

recipe(wav2vec2): add jakeBland/wav2vec-vm-finetune (audio-classification)#1114
codykk wants to merge 1 commit into
mainfrom
yongyue/add-jakeBland-wav2vec-vm-finetune-recipe

Conversation

@codykk

@codykk codykk commented Jul 15, 2026

Copy link
Copy Markdown

Add audio-classification recipes for jakeBland/wav2vec-vm-finetune

Adds fp16, w8a16, and w8a8 quantized recipe variants for jakeBland/wav2vec-vm-finetune (Wav2Vec2ForSequenceClassification, audio-classification task). The w8a8 variant delivers a 2.1x speedup and 71% memory reduction vs the fp32 baseline on CPU.


1. Recipe path(s)

  • examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_fp16_config.json
  • examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_w8a16_config.json
  • examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_w8a8_config.json

2. README row

Added jakeBland/wav2vec-vm-finetune | audio-classification to examples/recipes/README.md.

3. Build output dir

temp/wav2vec_vm_fp16/, temp/wav2vec_vm_w8a16/, temp/wav2vec_vm_w8a8/

4. Build log

All three variants: ✅ Build complete in ~70-77s

5. Appended findings

No methodology friction observed. Model uses standard wav2vec2 architecture fully covered by winml's existing Wav2Vec2OnnxConfig.

6. Optimum-coverage probe

{vendor: [], after_winml: ['audio-classification', 'audio-frame-classification', 'audio-xvector', 'automatic-speech-recognition', 'feature-extraction'], added_by_winml: [...all...], verdict: 'WINML-ONLY'}

7. Claimed (Effort, Goal, Outcome)

  • Effort: L0★ (new recipe, no source edits)
  • Goal ceiling: L1 (build + perf)
  • Outcome: L0 (recipe + README row)

8. Goal-ladder verdict table

Tier Verdict Evidence
L0 PASS All three recipes build successfully (fp16: 69.8s, w8a16: 77.0s, w8a8: 72.6s)
L1 PASS All three run on CPU EP without crash (see perf table below)

9. Methodology-evolution declaration

No methodology friction observed.

10. Perf & eval data

Baseline (fp32, no recipe, origin/main @ 2a2ff548):

EP / Device Precision Verdict Mean p50 Throughput RAM Δ Model Size
CPUExecutionProvider / cpu fp32 PASS 87.69 ms 87.57 ms 11.40 samples/s +1268 MB 1.2 GB

Recipe variants:

EP / Device Precision Verdict Mean p50 Throughput RAM Δ Model Size
CPUExecutionProvider / cpu fp16 PASS 91.09 ms 90.82 ms 10.98 samples/s +1279 MB 602 MB
CPUExecutionProvider / cpu w8a16 PASS 116.09 ms 114.76 ms 8.61 samples/s +464 MB 303 MB
CPUExecutionProvider / cpu w8a8 PASS 42.12 ms 41.67 ms 23.74 samples/s +373 MB 303 MB

Delta vs baseline:

  • w8a8: 2.1x faster, 75% smaller model, 71% less RAM
  • w8a16: 75% smaller model, 63% less RAM (latency higher due to dequantize overhead on CPU)
  • fp16: 50% smaller model (latency similar on CPU which doesn't natively accelerate fp16)

11. Component / op-level data

N/A — runtime check rules not available on this host (no parquet files).

12. Reproducible commands

# Baseline (fp32, no recipe)
winml build -m jakeBland/wav2vec-vm-finetune -o temp/baseline_wav2vec_vm --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild
winml perf -m temp/baseline_wav2vec_vm/model.onnx --device cpu --ep cpu

# fp16
winml build -c examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_fp16_config.json -m jakeBland/wav2vec-vm-finetune -o temp/wav2vec_vm_fp16/ --rebuild
winml perf -m temp/wav2vec_vm_fp16/model.onnx --device cpu --ep cpu

# w8a16
winml build -c examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_w8a16_config.json -m jakeBland/wav2vec-vm-finetune -o temp/wav2vec_vm_w8a16/ --rebuild
winml perf -m temp/wav2vec_vm_w8a16/model.onnx --device cpu --ep cpu

# w8a8
winml build -c examples/recipes/jakeBland_wav2vec-vm-finetune/cpu/cpu/audio-classification_w8a8_config.json -m jakeBland/wav2vec-vm-finetune -o temp/wav2vec_vm_w8a8/ --rebuild
winml perf -m temp/wav2vec_vm_w8a8/model.onnx --device cpu --ep cpu

Add fp16, w8a16, and w8a8 quantized recipe variants for the
jakeBland/wav2vec-vm-finetune model (Wav2Vec2ForSequenceClassification).

Tested on CPU EP with measurable delta vs fp32 baseline:
- fp16: 602MB (50% size reduction), 91ms avg latency
- w8a16: 303MB (75% size reduction), 116ms avg latency
- w8a8: 303MB (75% size reduction), 42ms avg (2.1x speedup)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codykk codykk changed the title Add audio-classification recipes for jakeBland/wav2vec-vm-finetune recipe(wav2vec2): add jakeBland/wav2vec-vm-finetune (audio-classification) Jul 15, 2026
@codykk codykk marked this pull request as ready for review July 15, 2026 07:30
@codykk codykk requested a review from a team as a code owner July 15, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant