Skip to content

Add an explicit --export-type (GENERIC | SPECIALIZED) selector for winml build instead of device+ep auto-routing #1090

Description

@DingmaomaoBJTU

Context

PR #1081 makes winml build emit an onnxruntime-genai bundle for a registered
decoder-LLM family (currently Qwen3). The trigger is intentionally implicit: a registered
family plus an explicit --ep qnn plus an NPU target (--device npu, or a
--device auto that resolves to the NPU). Every other target keeps the stock
single/composite build.

So a single family already has two possible outputs, selected today only by the
device+ep pair:

  • a single-model / composite ONNX build (today's stock output), and
  • a runtime-specialized multi-component bundle — the onnxruntime-genai layout
    (ctx / iter / embeddings / lm_head + genai_config.json + tokenizer), produced
    from a registered recipe.

Note both outputs are already target-aware (the stock build does device-aware quantization
and can compile an EPContext for a specific EP). The real distinction is output topology
and consuming runtime
— one ONNX model for ORT single-shot inference vs. a bundle for the
onnxruntime-genai generation loop — not "portable vs target-specific."

Suggestion (from review of #1081)

@xieofxie suggested (review threads on src/winml/modelkit/commands/build.py:
3555879362,
3555913471)
that instead of auto-routing on device+ep, we expose an explicit selector for the
kind of output, rather than hiding one output behind a device/ep combination:

So in the future if we have more recipes for each, user could easily switch instead of
automatically switch and hide another.

Proposal: an explicit --export-type selector

Add an explicit export-type selector on winml build:

--export-type GENERIC | SPECIALIZED
  • GENERIC (default) — the stock single-model / composite ONNX build. This is today's
    output for every target, so the default preserves current behavior.
  • SPECIALIZED — a runtime-specialized export produced from a registered recipe (today:
    the onnxruntime-genai NPU bundle).

--device / --ep / --precision remain target descriptors within the chosen
export-type; they no longer silently decide which kind of artifact is produced.

The point: an explicit selector scales better than a device+ep pair once a family has more
than one output, and gives users the choice without hidden routing.

Naming — open. --export-type overloads "export": there is already a winml export
command and an export stage inside the build pipeline, so --export-type may read as
"the export stage's type." Alternatives to weigh: --output-kind, --artifact,
--profile, --build-type, or a registry-keyed --recipe (see Binary vs. extensible
below). GENERIC/SPECIALIZED value casing / case-insensitivity is an implementation
detail.

Key decision: does SPECIALIZED infer the target, or require it?

The recipe already knows the target it needs (today: QNN NPU). Two options:

  1. Infer (recommended). winml build -m <model> -o out --export-type SPECIALIZED
    selects the recipe's target automatically. --device/--ep are only needed to pick
    among a recipe's supported targets, and an explicit contradiction
    (e.g. --device cpu) or unavailable hardware is an error. This makes the explicit
    selector genuinely more ergonomic than the device+ep trigger it replaces.
  2. Require. The user must still pass the recipe's --device/--ep (e.g. --ep qnn
    on an NPU); a mismatch is an error. Closest to today's behavior but offers little over
    the implicit trigger.

This is the central UX decision and is currently unspecified. --device auto must be
defined under either option: it should resolve to the recipe's supported target if one is
available on the host, else error.

Validation: SPECIALIZED with an unmet target fails fast (hard error)

When --export-type SPECIALIZED is passed but the target cannot serve the selected recipe
— today the only recipe is the QNN NPU bundle, so an explicit non-QNN --ep, a
non-NPU --device, or an NPU that is unavailable on the host — the build must abort with
a clear error and a non-zero exit
.

  • This is a fail-fast error (a UsageError, like the existing -c/--use-cache
    rejections), not a soft warning that continues.
  • It must not silently fall back to GENERIC or build a mismatched artifact.

The same fail-fast applies when the family has no specialized recipe at all. Because the
user asked for SPECIALIZED explicitly, an unmet target is a user error to surface, not
something to auto-correct.

Prerequisite: recipes must declare their supported targets

The validation above is not expressible today: GenaiBundleRecipe
(models/winml/genai_bundle.py) has no ep/device field — QNN+NPU is hardcoded at the call
site (build_genai_bundle(..., ep="qnn", device="npu")) and as orchestrator defaults. This
work must first add a declarative supported-targets field to the recipe (e.g.
supported_targets: tuple[(ep, device), ...]) so build can validate --export-type SPECIALIZED against it and drive target inference, keeping the routing architecture-agnostic.

Inherited constraints under SPECIALIZED

SPECIALIZED keeps the recipe-driven pipeline, so the controls the fast path already
rejects still apply and must error consistently: -c/--config, --use-cache,
--quant/--no-quant, --optimize/--no-optimize, --analyze/--no-analyze,
--compile/--no-compile, --max-optim-iterations, --allow-unsupported-nodes.
--precision remains the one supported transformer knob.

Binary vs. extensible axis

GENERIC/SPECIALIZED is a binary, but the stated motivation is scaling to multiple
recipes per family. Two specialized recipes for one family would make SPECIALIZED
ambiguous again — the same problem, one level up. Decide whether the axis is a fixed 2-value
enum plus a later recipe sub-selector, or an extensible, registry-keyed value from the
start (e.g. --export-type <recipe-name> / a dedicated --recipe), so the primary flag
names the concrete recipe when a family has more than one.

Interaction with the current device+ep trigger

--export-type is the explicit selector and overrides routing when passed. When
omitted, the existing --ep qnn + NPU trigger from #1081 is kept as a convenience
shortcut
for SPECIALIZED, so today's opt-in, non-destructive behavior is unchanged and
no existing command breaks. --export-type GENERIC forces the stock build even for a
registered family on the NPU.

Reconcile with perf (two vocabularies)

perf already selects the engine with --runtime winml | winml-genai
(commands/perf.py). Under this proposal a user builds --export-type SPECIALIZED then
benchmarks --runtime winml-genai — two different vocabularies for the same bundle.
Options: (a) share one vocabulary across build/perf/export; (b) keep --export-type
(build artifact) and --runtime (inference engine) as deliberately distinct axes and
document the mapping; or (c) have perf auto-detect a genai bundle from its directory
layout so --runtime is rarely needed. This is a decision, not just a note — the original
issue proposed winml | winml-genai precisely for this symmetry.

Discoverability

There is no way today to learn that a model supports SPECIALIZED. Consider surfacing the
supported export-types (and their targets) in winml inspect / analyze, and/or having the
GENERIC build emit a hint when the family also has a specialized recipe.

Why it's not in #1081

The device+ep trigger was locked with the product owner for that PR: it is opt-in
(--ep qnn must be explicit), non-destructive (every other target is unchanged), and needs
no new surface. Adding an explicit --export-type flag is a broader CLI-surface design
change, so it's better evaluated on its own.

Proposed follow-up

  • Add --export-type GENERIC | SPECIALIZED as the explicit output selector for build,
    defaulting to GENERIC; settle the flag name (see Naming).
  • Decide infer vs. require for SPECIALIZED targets, including --device auto.
  • Add a declarative supported-targets field to GenaiBundleRecipe so target validation
    and inference are data-driven (prerequisite).
  • Fail-fast (non-zero exit, UsageError) when SPECIALIZED can't serve the target or the
    family has no recipe — never silently fall back to GENERIC.
  • Keep the --ep qnn + NPU trigger as a shortcut for SPECIALIZED when --export-type is
    omitted; --export-type overrides it when passed.
  • Preserve the existing SPECIALIZED control rejections (config/cache/quant/optimize/etc.).
  • Decide binary enum vs. registry-keyed recipe value for multi-recipe families.
  • Reconcile with perf's --runtime winml | winml-genai (shared vocab, documented mapping,
    or perf auto-detect).
  • Surface supported export-types for discoverability (inspect/analyze, or a build hint).

Tracking the design discussion here so #1081 can merge with the current implicit trigger.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions