Skip to content
56 changes: 36 additions & 20 deletions docs/commands/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ $ winml build [options]
|---|---|---|---|---|
| `--config` | `-c` | path | `None` | `WinMLBuildConfig` JSON file, generated by `winml config`. If omitted, config is auto-generated from `-m`. |
| `--model` | `-m` | string | `None` | Hugging Face model ID or path to an existing `.onnx` file. |
| `--export-type` | | choice | `generic` | Output selector: `generic` builds the stock single/composite ONNX model; `optimized` builds the family's registered runtime-optimized recipe (today the onnxruntime-genai NPU bundle), inferring its target. `optimized` fails fast if the family has no recipe or `--ep`/`--device` contradicts it. |
| `--output-dir` | `-o` | path | `None` | Directory for all build artifacts. Mutually exclusive with `--use-cache`. |
| `--use-cache/--no-use-cache` | | flag | `false` | Store artifacts in the winml-cli global cache (`~/.cache/winml/`). Mutually exclusive with `--output-dir`. |
| `--rebuild/--no-rebuild` | | flag | `false` | Overwrite existing artifacts and re-run the full pipeline. |
Expand Down Expand Up @@ -58,11 +59,10 @@ single-pass build. Individual stages can be suppressed with `--no-quant`,

## Genai bundles for decoder LLMs (NPU + QNN)

For a registered decoder-LLM family (currently **Qwen3**), targeting the NPU HTP
with an explicit `--ep qnn` switches `winml build` to produce a complete
[onnxruntime-genai](https://github.com/microsoft/onnxruntime-genai) **bundle**
directory instead of the stock per-model ONNX output. Pair `--ep qnn` with
`--device npu`, or with `--device auto` when auto-detection resolves to the NPU:
For a registered decoder-LLM family (currently **Qwen3**), `--export-type
optimized` switches `winml build` from the stock per-model ONNX output to a
complete [onnxruntime-genai](https://github.com/microsoft/onnxruntime-genai)
**bundle** directory:

| File | Role | Precision |
|---|---|---|
Expand All @@ -73,20 +73,36 @@ directory instead of the stock per-model ONNX output. Pair `--ep qnn` with
| `genai_config.json` + tokenizer | onnxruntime-genai runtime metadata | — |

```bash
# One command: HF decoder LLM → full onnxruntime-genai bundle on the NPU
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --device npu --ep qnn
# or let device auto-detection pick the NPU:
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --device auto --ep qnn
# One command: HF decoder LLM → full onnxruntime-genai bundle
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --export-type optimized
```

The trigger is data-driven and opt-in: **`--ep qnn` must be passed explicitly**,
alongside an NPU target (`--device npu`, or `--device auto` resolving to the NPU).
Every other target — including Qwen3 on CPU/GPU, or an auto-detected NPU target
without an explicit `--ep qnn` — keeps the existing behavior (the stock composite
build). `--output-dir` is required (the bundle is a directory) and `--use-cache`
is not supported. Use `--precision` to override the transformer precision; the CPU
companions always use their bundle-standard precisions. See
[Qwen3 — Genai Bundle](../samples/qwen3-genai-bundle.md) for the full walkthrough.
`--export-type optimized` **infers** the recipe's target (Qwen3 builds for the
NPU HTP via QNN), so `--ep`/`--device` are optional — and, unlike the shortcut
below, the optimized path does not gate on detecting the target device locally.
If you do pin `--ep`/`--device`, they must match the recipe: a contradicting
value, an unregistered family, a pre-exported `.onnx` input, or module mode all
fail fast rather than silently falling back to a generic build. `--output-dir` is
required (the bundle is a directory) and `--use-cache` is not supported. Use
`--precision` to override the transformer precision; the CPU companions always
use their bundle-standard precisions.

!!! note "Backward-compatible shortcut"
When `--export-type` is omitted, a registered family with an explicit
`--ep qnn` on an NPU target still routes to its optimized bundle. The NPU
target may be explicit (`--device npu`) or resolved from `auto` — whether
`--device auto` is typed or `--device` is left off entirely:

```bash
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --device npu --ep qnn
# or let device auto-detection pick the NPU (--device may be omitted):
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --ep qnn
```

`--export-type generic` always forces the stock composite build, even for a
registered family on the NPU.

See [Qwen3 — Genai Bundle](../samples/qwen3-genai-bundle.md) for the full walkthrough.

## Examples

Expand Down Expand Up @@ -150,9 +166,9 @@ winml build -m microsoft/resnet-50 -o output/ \
pre-exported ONNX inputs because the export step has already happened.
- **Existing artifacts are reused by default.** Pass `--rebuild` to force a
fresh run after changing the config.
- **Genai bundles require `--output-dir`, not `--use-cache`.** A decoder-LLM
bundle (`--device npu --ep qnn`) writes a directory of components and rejects
`--use-cache`.
- **Genai bundles require `--output-dir`, not `--use-cache`.** An optimized
decoder-LLM bundle (`--export-type optimized`, or the `--device npu --ep qnn`
shortcut) writes a directory of components and rejects `--use-cache`.

## See also

Expand Down
34 changes: 22 additions & 12 deletions docs/samples/qwen3-genai-bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ memory-bound companions stay on CPU.

```mermaid
graph LR
A["winml build -m Qwen/Qwen3-0.6B --device npu --ep qnn"] --> B[Genai bundle recipe]
A["winml build -m Qwen/Qwen3-0.6B --export-type optimized"] --> B[Genai bundle recipe]
B --> C[ctx.onnx / iter.onnx — NPU]
B --> D[embeddings.onnx — CPU]
B --> E[lm_head.onnx — CPU]
Expand All @@ -41,14 +41,12 @@ graph LR

## Step 1: Build the bundle (one command)

Targeting the NPU HTP with an explicit `--ep qnn` switches `winml build` from the
stock per-model ONNX output to the full genai bundle. Pair it with `--device npu`,
or `--device auto` when auto-detection resolves to the NPU:
`--export-type optimized` switches `winml build` from the stock per-model ONNX
output to the full genai bundle. It **infers** the recipe's target (Qwen3 builds
for the NPU HTP via QNN), so `--ep`/`--device` are optional:

```bash
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --device npu --ep qnn
# or, letting device auto-detection pick the NPU:
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --device auto --ep qnn
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --export-type optimized
```

This builds (or reuses from cache) all four components and assembles them, writing
Expand All @@ -63,11 +61,23 @@ The CPU companions likewise keep their bundle-standard precisions.

Force a clean rebuild of every component with `--rebuild`.

!!! note "Opt-in, non-destructive"
The bundle path only triggers on an **explicit `--ep qnn`** together with an
NPU target — either `--device npu` or a `--device auto` that resolves to the
NPU. Qwen3 on any other target — CPU, GPU, or an auto-detected NPU *without*
an explicit `--ep qnn` — still produces the stock composite build, unchanged.
!!! note "Backward-compatible shortcut"
When `--export-type` is omitted, an **explicit `--ep qnn`** targeting the NPU
still routes a registered family to its bundle. The NPU target may be
explicit (`--device npu`) or resolved from `auto` — whether `--device auto`
is typed or `--device` is omitted:

```bash
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --device npu --ep qnn
# or, letting device auto-detection pick the NPU (--device may be omitted):
winml build -m Qwen/Qwen3-0.6B -o out/qwen3-bundle --ep qnn
```

Qwen3 on any other target — CPU, GPU, or an auto-detected NPU *without* an
explicit `--ep qnn` — still produces the stock composite build. `--export-type
generic` always forces the stock build, even on the NPU. A pinned
`--ep`/`--device` that contradicts the recipe fails fast rather than silently
reverting.

## Step 2: Tune context and prefill lengths (optional)

Expand Down
Loading
Loading