Skip to content

Add --export-type (generic | optimized) selector to winml build#1104

Open
DingmaomaoBJTU wants to merge 7 commits into
mainfrom
dingmaomaobjtu-add-export-type-build-flag
Open

Add --export-type (generic | optimized) selector to winml build#1104
DingmaomaoBJTU wants to merge 7 commits into
mainfrom
dingmaomaobjtu-add-export-type-build-flag

Conversation

@DingmaomaoBJTU

@DingmaomaoBJTU DingmaomaoBJTU commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Closes #1090.

Summary

Replaces the implicit device+ep auto-routing to the onnxruntime-genai bundle with an explicit --export-type selector on winml build, so choosing between a stock ONNX export and a runtime-optimized bundle is a deliberate, discoverable decision instead of a side effect of --ep qnn + NPU.

--export-type Result
generic (default) Stock single/composite ONNX build (unchanged).
optimized The family's registered runtime-optimized recipe — today the onnxruntime-genai NPU bundle.

Behavior

  • optimized infers the target. The recipe declares its supported_targets ((ep, device) pairs), so --ep/--device are optional; the first target is inferred (Qwen3 → qnn/npu). Building a bundle is hardware-independent, so this path does not probe the local device.
  • Fail-fast, no silent fallback. optimized errors when the family has no recipe, the input is a pre-exported .onnx, it is module mode, or an explicitly pinned --ep/--device contradicts the recipe.
  • Backward compatible. When --export-type is omitted, the existing shortcut still applies: a registered family with an explicit --ep qnn on an NPU target (--device npu, or --device auto resolving to the NPU) routes to its bundle. --export-type generic always forces the stock build, even on the NPU.

Changes

  • Recipe data model (genai_bundle.py): new GenaiTarget frozen dataclass and a required supported_targets field on GenaiBundleRecipe, validated (non-empty) at registration. New symbols exported from the package __init__.
  • Qwen3 recipe: declares supported_targets=(GenaiTarget(ep="qnn", device="npu"),).
  • winml build (build.py): adds the --export-type choice option and refactors the routing into small helpers (_resolve_genai_recipe, _resolve_optimized_target, _maybe_build_genai_bundle) that keep the switch data-driven and architecture-agnostic.
  • Docs: docs/commands/build.md (flag row + reframed genai-bundle section) and docs/samples/qwen3-genai-bundle.md now lead with --export-type optimized and document the shortcut as backward-compatible.

Sample

Build the optimized onnxruntime-genai NPU bundle for a decoder LLM (the target is inferred from the recipe, so --ep/--device are optional):

winml build -m Qwen/Qwen3-0.6B -o out/ --export-type optimized

Force a stock ONNX build instead:

winml build -m Qwen/Qwen3-0.6B -o out/ --export-type generic

The selector is discoverable in --help:

--export-type [generic|optimized]
                                Output selector. 'generic' builds the stock
                                single/composite ONNX model. 'optimized'
                                builds the registered runtime-optimized recipe
                                for the family (today: the onnxruntime-genai
                                NPU bundle); it infers the recipe's target and
                                errors if the family has no recipe or
                                --ep/--device contradicts it. ...

Fail-fast validation when the model type has no optimized recipe (verified in terminal):

$ winml build -m microsoft/resnet-50 -o out --export-type optimized
Error: --export-type optimized: no optimized recipe is registered for model type 'resnet'.

Tests

New CLI routing tests (infer-and-build, generic forces stock, unregistered-family error) plus focused unit tests for target inference/contradiction and the optimized preconditions; registry tests for the empty-supported_targets guard. The genai build/routing suites (tests/unit/commands/test_build_genai_bundle.py and tests/unit/models/winml/) pass; ruff check and strict mypy clean.

Replace the implicit device+ep auto-routing to the genai bundle with an
explicit --export-type flag (issue #1090):

- generic (default): stock single/composite ONNX build.
- specialized: builds the family's registered runtime-specialized recipe
  (today the onnxruntime-genai NPU bundle), inferring the recipe's target.
  Fails fast when the family has no recipe, the input is a pre-exported
  .onnx, it is module mode, or --ep/--device contradicts the recipe.

Recipes now declare supported_targets ((ep, device) pairs, validated at
registration). The specialized path infers the first target when the user
pins neither --ep nor --device, and building a bundle stays
hardware-independent (no device probing).

Omitting --export-type preserves the backward-compatible shortcut: a
registered family with an explicit --ep qnn on an NPU target still routes
to its bundle. --export-type generic always forces the stock build.
@DingmaomaoBJTU DingmaomaoBJTU requested a review from a team as a code owner July 13, 2026 09:15
Comment thread src/winml/modelkit/commands/build.py Outdated
github-actions Bot added 2 commits July 14, 2026 15:08
Resolve build.py conflict by keeping both the new --export-type option
and #1083's dynamic export controls (--shape-config/--input-specs/
--export-config/--dynamic-axes), in the option decorators and the build()
signature.
@DingmaomaoBJTU DingmaomaoBJTU changed the title Add --export-type (generic | specialized) selector to winml build Add --export-type (generic | optimized) selector to winml build Jul 14, 2026
# model-specific value lives in the recipe. ``--export-type generic`` (or
# any other device/ep combination without the flag) keeps the stock
# single/composite build.
if _maybe_build_genai_bundle(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This optimized path is still reached only after the earlier if ep is None: resolve_check_device_ep(...) block has run. That means winml build -m Qwen/Qwen3-0.6B -o out --export-type optimized still probes local hardware and can fail before the recipe target is inferred, which contradicts the new contract/docs that optimized builds are hardware-independent and don't require detecting the NPU locally. Can we route explicit --export-type optimized before auto-resolving device/EP, or skip that resolution when the optimized target will be inferred from the recipe?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in ffc3af9. --export-type optimized now skips the resolve_check_device_ep auto-EP probe entirely: the optimized branch resolves (ep, device) from the recipe via _resolve_optimized_target, so the build no longer depends on detecting the accelerator locally. Added a regression test that makes resolve_check_device_ep raise and asserts the bundle still builds (and the probe is not called).

The implicit optimized-bundle shortcut required --device to be explicitly provided (is_cli_provided), so omitting it (default `auto`) behaved differently from typing --device auto: --ep qnn alone fell through to the generic pipeline instead of the NPU bundle.

Drop the --device provided-guard so the omitted `auto` default is resolved by the existing auto-resolution below, matching explicit --device auto. The --ep guard is kept (a bare --device npu can be served by non-QNN EPs, so it intentionally does not route).

Add regression tests for --ep qnn without --device (routes when auto->npu, falls through when auto resolves to non-npu) and update the shortcut docstring plus build/sample docs.
@xieofxie

xieofxie commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

For your description

Build the optimized onnxruntime-genai NPU bundle for a decoder LLM (the target is inferred from the recipe, so --ep/--device are optional):

If it is the case, I suggest we resolve first and then error that the --export-type is not supported for ep=xx, device=xx

Infer target from recipe is bad due to

  1. conflict with ep device flags resolving
  2. if we have more optimized recipes, it is hard to resolve

An earlier `if ep is None: resolve_check_device_ep(...)` block probed the host and picked an EP before the genai-bundle routing. That made `winml build -m <model> --export-type optimized` -- which infers its target from the recipe and is hardware-independent by contract -- still probe local hardware and fail on a machine without the recipe's accelerator (e.g. no NPU) before the recipe target was ever inferred.

Skip that auto-EP resolution when --export-type optimized is explicitly requested; the optimized branch resolves (ep, device) from the recipe via _resolve_optimized_target. Add a regression test asserting the probe is not called (and the bundle still builds) even when resolve_check_device_ep would raise.

Addresses review feedback on PR #1104.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an explicit --export-type (GENERIC | SPECIALIZED) selector for winml build instead of device+ep auto-routing

2 participants