genai --compile: model load can native-crash when compile and load share one process; isolate og.Model load into its own subprocess

[DingmaomaoBJTU]

Summary

winml perf -m <bundle> --runtime winml-genai --device npu --compile can silently native-crash at model load (before the first token) even when all EPContext stages are already compiled and cached. The failure is not a wedged HTP and not the model itself — it is the --compile code path running the crash-prone stage compilation and the og.Model() load in the same parent process.

Reproduced independently on two setups (Windows ARM64 / QNN HTP).

Root cause

In GenaiSession.load() (src/winml/modelkit/session/genai_session.py), --compile (compile=True) does both of these in one process:

1. _prepare_compiled_bundle() — compiles each EPContext-capable stage in a spawned subprocess. QNN can native-crash during interpreter/driver teardown of those subprocesses (PR feat(build): one-command Qwen3 genai bundle via winml build (npu + qnn) #1081 adds an EPContext salvage that recovers the compiled artifact, but the native fault still happened in-session).
2. Immediately afterward, og.Model(og.Config(_compiled/)) is created in the same parent process to run generation.

The compile orchestration appears to leave the parent process / native QNN state fragile, so the subsequent og.Model() load faults (python.exe faulting, 0xC0000005 / 0xC0000374) with no Python traceback.

Evidence it is process-isolation, not the bundle

Loading the already-compiled, self-contained _compiled/ directory directly in a fresh process (no --compile, so no _prepare_compiled_bundle()) generates cleanly:

winml perf -m out/qwen3-bundle/_compiled --runtime winml-genai --device npu \
  --max-new-tokens 20 --prompt "What is the capital of France?"

Result: winml-genai / qnn / npu, 15-token prompt -> 20 tokens, TTFT ~537 ms (P50 531), decode ~5.0 tok/s, 10 iterations, results JSON saved. The same _compiled/ bundle that faults under --compile runs fine when loaded on its own.

_mirror_non_onnx_files() already skips files that exist (if dst.exists(): continue), so this is not re-mirror memory pressure — it is specifically compile + load sharing a process.

Workaround (already documented)

docs/samples/qwen3-genai-bundle.md now describes the two-step flow: run --compile once to produce <bundle>/_compiled/, then run winml perf -m <bundle>/_compiled (no --compile) in a fresh process. The second command loads the compiled EPContext graphs directly, so it does not JIT-compile anything.

Proposed fix (root cause)

In --compile mode, after _prepare_compiled_bundle() returns, load and run og.Model() in its own isolated subprocess (spawn / re-exec) so the crash-prone compile orchestration cannot poison the process that performs generation. Equivalent options to weigh:

• Have winml perf --compile internally delegate generation to a fresh child process pointed at the prepared _compiled/ directory.
• Or split the public flow so compilation and generation are never done in one process.

Acceptance: winml perf -m <bundle> --compile --runtime winml-genai --device npu produces tokens end-to-end in a single command, without a manual second invocation, even when a stage compile teardown-crashes.

Related

• PR feat(build): one-command Qwen3 genai bundle via winml build (npu + qnn) #1081 (adds the Qwen3 genai bundle build + the EPContext salvage that makes each --compile run recover completed stages; commit cfc3e015).

[DingmaomaoBJTU]

Triage Summary

Priority: P1 (crash in core perf workflow on NPU)
Component: perf / genai-session (process isolation)
Assigned: @DingmaomaoBJTU (author, best context)

Assessment

• Native crash (0xC0000005 / 0xC0000374) during \og.Model()\ load after in-process --compile\ on QNN HTP.
• Root cause well-identified: compile subprocess teardown leaves parent process/QNN state fragile.
• Proposed fix (spawn \og.Model()\ load in isolated subprocess) is reasonable and scoped.
• Workaround exists: run perf on the _compiled/\ directory without --compile\ flag.

Triage Decision

Approve as P1 — impacts the happy-path NPU perf workflow. Fix should isolate model load into a subprocess as proposed in the issue body. The existing workaround (two-step: compile then perf separately) is acceptable for unblocking users in the interim.

---

Triaged: 2026-07-13