Context
PR #1081 makes winml build emit an onnxruntime-genai bundle for a registered
decoder-LLM family (currently Qwen3). The trigger is intentionally implicit: a registered
family plus an explicit --ep qnn plus an NPU target (--device npu, or a
--device auto that resolves to the NPU). Every other target keeps the stock
single/composite build.
So a single family already has two possible outputs, selected today only by the
device+ep pair:
- a single-model / composite ONNX build (today's stock output), and
- a runtime-specialized multi-component bundle — the onnxruntime-genai layout
(ctx / iter / embeddings / lm_head + genai_config.json + tokenizer), produced
from a registered recipe.
Note both outputs are already target-aware (the stock build does device-aware quantization
and can compile an EPContext for a specific EP). The real distinction is output topology
and consuming runtime — one ONNX model for ORT single-shot inference vs. a bundle for the
onnxruntime-genai generation loop — not "portable vs target-specific."
Suggestion (from review of #1081)
@xieofxie suggested (review threads on src/winml/modelkit/commands/build.py:
3555879362,
3555913471)
that instead of auto-routing on device+ep, we expose an explicit selector for the
kind of output, rather than hiding one output behind a device/ep combination:
So in the future if we have more recipes for each, user could easily switch instead of
automatically switch and hide another.
Proposal: an explicit --export-type selector
Add an explicit export-type selector on winml build:
--export-type GENERIC | SPECIALIZED
- GENERIC (default) — the stock single-model / composite ONNX build. This is today's
output for every target, so the default preserves current behavior.
- SPECIALIZED — a runtime-specialized export produced from a registered recipe (today:
the onnxruntime-genai NPU bundle).
--device / --ep / --precision remain target descriptors within the chosen
export-type; they no longer silently decide which kind of artifact is produced.
The point: an explicit selector scales better than a device+ep pair once a family has more
than one output, and gives users the choice without hidden routing.
Naming — open. --export-type overloads "export": there is already a winml export
command and an export stage inside the build pipeline, so --export-type may read as
"the export stage's type." Alternatives to weigh: --output-kind, --artifact,
--profile, --build-type, or a registry-keyed --recipe (see Binary vs. extensible
below). GENERIC/SPECIALIZED value casing / case-insensitivity is an implementation
detail.
Key decision: does SPECIALIZED infer the target, or require it?
The recipe already knows the target it needs (today: QNN NPU). Two options:
- Infer (recommended).
winml build -m <model> -o out --export-type SPECIALIZED
selects the recipe's target automatically. --device/--ep are only needed to pick
among a recipe's supported targets, and an explicit contradiction
(e.g. --device cpu) or unavailable hardware is an error. This makes the explicit
selector genuinely more ergonomic than the device+ep trigger it replaces.
- Require. The user must still pass the recipe's
--device/--ep (e.g. --ep qnn
on an NPU); a mismatch is an error. Closest to today's behavior but offers little over
the implicit trigger.
This is the central UX decision and is currently unspecified. --device auto must be
defined under either option: it should resolve to the recipe's supported target if one is
available on the host, else error.
Validation: SPECIALIZED with an unmet target fails fast (hard error)
When --export-type SPECIALIZED is passed but the target cannot serve the selected recipe
— today the only recipe is the QNN NPU bundle, so an explicit non-QNN --ep, a
non-NPU --device, or an NPU that is unavailable on the host — the build must abort with
a clear error and a non-zero exit.
- This is a fail-fast error (a
UsageError, like the existing -c/--use-cache
rejections), not a soft warning that continues.
- It must not silently fall back to
GENERIC or build a mismatched artifact.
The same fail-fast applies when the family has no specialized recipe at all. Because the
user asked for SPECIALIZED explicitly, an unmet target is a user error to surface, not
something to auto-correct.
Prerequisite: recipes must declare their supported targets
The validation above is not expressible today: GenaiBundleRecipe
(models/winml/genai_bundle.py) has no ep/device field — QNN+NPU is hardcoded at the call
site (build_genai_bundle(..., ep="qnn", device="npu")) and as orchestrator defaults. This
work must first add a declarative supported-targets field to the recipe (e.g.
supported_targets: tuple[(ep, device), ...]) so build can validate --export-type SPECIALIZED against it and drive target inference, keeping the routing architecture-agnostic.
Inherited constraints under SPECIALIZED
SPECIALIZED keeps the recipe-driven pipeline, so the controls the fast path already
rejects still apply and must error consistently: -c/--config, --use-cache,
--quant/--no-quant, --optimize/--no-optimize, --analyze/--no-analyze,
--compile/--no-compile, --max-optim-iterations, --allow-unsupported-nodes.
--precision remains the one supported transformer knob.
Binary vs. extensible axis
GENERIC/SPECIALIZED is a binary, but the stated motivation is scaling to multiple
recipes per family. Two specialized recipes for one family would make SPECIALIZED
ambiguous again — the same problem, one level up. Decide whether the axis is a fixed 2-value
enum plus a later recipe sub-selector, or an extensible, registry-keyed value from the
start (e.g. --export-type <recipe-name> / a dedicated --recipe), so the primary flag
names the concrete recipe when a family has more than one.
Interaction with the current device+ep trigger
--export-type is the explicit selector and overrides routing when passed. When
omitted, the existing --ep qnn + NPU trigger from #1081 is kept as a convenience
shortcut for SPECIALIZED, so today's opt-in, non-destructive behavior is unchanged and
no existing command breaks. --export-type GENERIC forces the stock build even for a
registered family on the NPU.
Reconcile with perf (two vocabularies)
perf already selects the engine with --runtime winml | winml-genai
(commands/perf.py). Under this proposal a user builds --export-type SPECIALIZED then
benchmarks --runtime winml-genai — two different vocabularies for the same bundle.
Options: (a) share one vocabulary across build/perf/export; (b) keep --export-type
(build artifact) and --runtime (inference engine) as deliberately distinct axes and
document the mapping; or (c) have perf auto-detect a genai bundle from its directory
layout so --runtime is rarely needed. This is a decision, not just a note — the original
issue proposed winml | winml-genai precisely for this symmetry.
Discoverability
There is no way today to learn that a model supports SPECIALIZED. Consider surfacing the
supported export-types (and their targets) in winml inspect / analyze, and/or having the
GENERIC build emit a hint when the family also has a specialized recipe.
Why it's not in #1081
The device+ep trigger was locked with the product owner for that PR: it is opt-in
(--ep qnn must be explicit), non-destructive (every other target is unchanged), and needs
no new surface. Adding an explicit --export-type flag is a broader CLI-surface design
change, so it's better evaluated on its own.
Proposed follow-up
- Add
--export-type GENERIC | SPECIALIZED as the explicit output selector for build,
defaulting to GENERIC; settle the flag name (see Naming).
- Decide infer vs. require for
SPECIALIZED targets, including --device auto.
- Add a declarative supported-targets field to
GenaiBundleRecipe so target validation
and inference are data-driven (prerequisite).
- Fail-fast (non-zero exit,
UsageError) when SPECIALIZED can't serve the target or the
family has no recipe — never silently fall back to GENERIC.
- Keep the
--ep qnn + NPU trigger as a shortcut for SPECIALIZED when --export-type is
omitted; --export-type overrides it when passed.
- Preserve the existing
SPECIALIZED control rejections (config/cache/quant/optimize/etc.).
- Decide binary enum vs. registry-keyed recipe value for multi-recipe families.
- Reconcile with
perf's --runtime winml | winml-genai (shared vocab, documented mapping,
or perf auto-detect).
- Surface supported export-types for discoverability (
inspect/analyze, or a build hint).
Tracking the design discussion here so #1081 can merge with the current implicit trigger.
Context
PR #1081 makes
winml buildemit an onnxruntime-genai bundle for a registereddecoder-LLM family (currently Qwen3). The trigger is intentionally implicit: a registered
family plus an explicit
--ep qnnplus an NPU target (--device npu, or a--device autothat resolves to the NPU). Every other target keeps the stocksingle/composite build.
So a single family already has two possible outputs, selected today only by the
device+eppair:(
ctx/iter/embeddings/lm_head+genai_config.json+ tokenizer), producedfrom a registered recipe.
Note both outputs are already target-aware (the stock build does device-aware quantization
and can compile an EPContext for a specific EP). The real distinction is output topology
and consuming runtime — one ONNX model for ORT single-shot inference vs. a bundle for the
onnxruntime-genai generation loop — not "portable vs target-specific."
Suggestion (from review of #1081)
@xieofxie suggested (review threads on
src/winml/modelkit/commands/build.py:3555879362,
3555913471)
that instead of auto-routing on
device+ep, we expose an explicit selector for thekind of output, rather than hiding one output behind a device/ep combination:
Proposal: an explicit
--export-typeselectorAdd an explicit export-type selector on
winml build:output for every target, so the default preserves current behavior.
the onnxruntime-genai NPU bundle).
--device/--ep/--precisionremain target descriptors within the chosenexport-type; they no longer silently decide which kind of artifact is produced.
The point: an explicit selector scales better than a device+ep pair once a family has more
than one output, and gives users the choice without hidden routing.
Key decision: does
SPECIALIZEDinfer the target, or require it?The recipe already knows the target it needs (today: QNN NPU). Two options:
winml build -m <model> -o out --export-type SPECIALIZEDselects the recipe's target automatically.
--device/--epare only needed to pickamong a recipe's supported targets, and an explicit contradiction
(e.g.
--device cpu) or unavailable hardware is an error. This makes the explicitselector genuinely more ergonomic than the device+ep trigger it replaces.
--device/--ep(e.g.--ep qnnon an NPU); a mismatch is an error. Closest to today's behavior but offers little over
the implicit trigger.
This is the central UX decision and is currently unspecified.
--device automust bedefined under either option: it should resolve to the recipe's supported target if one is
available on the host, else error.
Validation:
SPECIALIZEDwith an unmet target fails fast (hard error)When
--export-type SPECIALIZEDis passed but the target cannot serve the selected recipe— today the only recipe is the QNN NPU bundle, so an explicit non-QNN
--ep, anon-NPU
--device, or an NPU that is unavailable on the host — the build must abort witha clear error and a non-zero exit.
UsageError, like the existing-c/--use-cacherejections), not a soft warning that continues.
GENERICor build a mismatched artifact.The same fail-fast applies when the family has no specialized recipe at all. Because the
user asked for
SPECIALIZEDexplicitly, an unmet target is a user error to surface, notsomething to auto-correct.
Prerequisite: recipes must declare their supported targets
The validation above is not expressible today:
GenaiBundleRecipe(
models/winml/genai_bundle.py) has no ep/device field — QNN+NPU is hardcoded at the callsite (
build_genai_bundle(..., ep="qnn", device="npu")) and as orchestrator defaults. Thiswork must first add a declarative supported-targets field to the recipe (e.g.
supported_targets: tuple[(ep, device), ...]) sobuildcan validate--export-type SPECIALIZEDagainst it and drive target inference, keeping the routing architecture-agnostic.Inherited constraints under
SPECIALIZEDSPECIALIZEDkeeps the recipe-driven pipeline, so the controls the fast path alreadyrejects still apply and must error consistently:
-c/--config,--use-cache,--quant/--no-quant,--optimize/--no-optimize,--analyze/--no-analyze,--compile/--no-compile,--max-optim-iterations,--allow-unsupported-nodes.--precisionremains the one supported transformer knob.Binary vs. extensible axis
GENERIC/SPECIALIZEDis a binary, but the stated motivation is scaling to multiplerecipes per family. Two specialized recipes for one family would make
SPECIALIZEDambiguous again — the same problem, one level up. Decide whether the axis is a fixed 2-value
enum plus a later recipe sub-selector, or an extensible, registry-keyed value from the
start (e.g.
--export-type <recipe-name>/ a dedicated--recipe), so the primary flagnames the concrete recipe when a family has more than one.
Interaction with the current device+ep trigger
--export-typeis the explicit selector and overrides routing when passed. Whenomitted, the existing
--ep qnn+ NPU trigger from #1081 is kept as a convenienceshortcut for
SPECIALIZED, so today's opt-in, non-destructive behavior is unchanged andno existing command breaks.
--export-type GENERICforces the stock build even for aregistered family on the NPU.
Reconcile with
perf(two vocabularies)perfalready selects the engine with--runtime winml | winml-genai(
commands/perf.py). Under this proposal a user builds--export-type SPECIALIZEDthenbenchmarks
--runtime winml-genai— two different vocabularies for the same bundle.Options: (a) share one vocabulary across
build/perf/export; (b) keep--export-type(build artifact) and
--runtime(inference engine) as deliberately distinct axes anddocument the mapping; or (c) have
perfauto-detect a genai bundle from its directorylayout so
--runtimeis rarely needed. This is a decision, not just a note — the originalissue proposed
winml | winml-genaiprecisely for this symmetry.Discoverability
There is no way today to learn that a model supports
SPECIALIZED. Consider surfacing thesupported export-types (and their targets) in
winml inspect/analyze, and/or having theGENERICbuild emit a hint when the family also has a specialized recipe.Why it's not in #1081
The device+ep trigger was locked with the product owner for that PR: it is opt-in
(
--ep qnnmust be explicit), non-destructive (every other target is unchanged), and needsno new surface. Adding an explicit
--export-typeflag is a broader CLI-surface designchange, so it's better evaluated on its own.
Proposed follow-up
--export-type GENERIC | SPECIALIZEDas the explicit output selector forbuild,defaulting to
GENERIC; settle the flag name (see Naming).SPECIALIZEDtargets, including--device auto.GenaiBundleRecipeso target validationand inference are data-driven (prerequisite).
UsageError) whenSPECIALIZEDcan't serve the target or thefamily has no recipe — never silently fall back to
GENERIC.--ep qnn+ NPU trigger as a shortcut forSPECIALIZEDwhen--export-typeisomitted;
--export-typeoverrides it when passed.SPECIALIZEDcontrol rejections (config/cache/quant/optimize/etc.).perf's--runtime winml | winml-genai(shared vocab, documented mapping,or perf auto-detect).
inspect/analyze, or a build hint).Tracking the design discussion here so #1081 can merge with the current implicit trigger.