[WebGPU EP] Support device-free compile-only sessions for offline graph transformation

[mingmingtasd]

Description

Enable a WebGPU EP session to run graph transformation (optimization + partitioning) and serialize the resulting model without creating a device or touching hardware. This supports offline / ahead-of-time model compilation in device-free or sandboxed host processes, where the optimized model is produced in one process and executed later in another.

WebGPU compile-only mode (new provider option
"ep.webgpuexecutionprovider.compileOnly"):

• webgpu_provider_options.h, webgpu_provider_factory.cc: parse the option.
• webgpu_context.cc/.h: in compile-only mode, skip Dawn adapter/device creation and all device-dependent initialization (queue, limits, features, adapter info, buffer/program managers, etc.), so the context can run graph transformation without a GPU. IsCompileOnly() exposes the state. Why: creating the Dawn device requires real hardware and is wasteful for a session that only produces a compiled model and never executes kernels.

Stop-before-finalize, derived from an EP capability (no new public API):

• execution_provider.h: add virtual IExecutionProvider::ShouldStopBeforeSessionStateFinalization() (default false); other EPs inherit the default and are unaffected.
• webgpu_execution_provider.cc/.h: override it to return the WebGpuContext compile-only state.
• inference_session.cc: after graph transformation, Resolve and EPContext model generation, if any registered EP requests it, return before session-state finalization (kernel creation, PrePack, initializer upload, memory planning). The session is left non-runnable; only the output/EPContext model is produced. Why: finalization creates kernels and allocates device memory, which both requires a device and is pure waste for a throwaway compile-only session. Deriving the stop from the EP's own compile-only state, rather than from a separate public session config key, adds no public API surface and makes the two properties inseparable: a compile-only WebGPU EP cannot be misconfigured into attempting finalization. It works identically whether the WebGPU EP is built in (--use_webgpu) or loaded as a plugin (--use_webgpu shared_lib), since both paths produce the same WebGpuExecutionProvider.

Lazy allocator initialization:

• allocator.cc/.h: GpuBufferAllocator computes mapped_at_creation on the first Alloc() instead of in the constructor. Why: the constructor previously queried the device (BufferManager::SupportsUMA), which is unavailable in compile-only mode. A compile-only session never allocates, so the query is deferred and never runs.

Enforce disable_cpu_ep_fallback before the compile-only early-return:

• inference_session.cc: move the "session.disable_cpu_ep_fallback" check to run right after graph transformation / Resolve, ahead of the stop-before-finalize early-return. Why: node-to-EP assignment is complete after partitioning, so the check has everything it needs. Running it before the early-return means the guarantee (no node silently falls back to the CPU EP) is enforced for compile-only sessions too, instead of being skipped.

Virtual GPU device when hardware enumeration is unavailable:

• ep/factory.cc/.h (adapter EP factory) and plugin_ep/ep_factory_webgpu.cc/.h (built-in EP factory): when no GPU OrtHardwareDevice is discovered and the environment sets "allow_virtual_devices=1", register a virtual GPU OrtEpDevice (vendor/device id 0, is_virtual=1) so the WebGPU EP can still be selected.
• environment.cc, plugin_ep/ep_factory_internal.h, plugin_ep/ep_factory_internal_impl.h: plumb the allow_virtual_devices flag from the environment to the internal factory before GetSupportedDevices runs. Why: on hosts where OS device enumeration is unavailable (e.g. a sandboxed process under Win32k lockdown), ORT device discovery is skipped and no GPU device is found, so the WebGPU EP would not be selectable. The flag is plumbed rather than read from the OrtEnv singleton inside GetSupportedDevices because internal EPs are registered while the OrtEnv creation mutex is held; querying the singleton there would re-enter that lock and deadlock.

Build: delay-load user32.dll (cmake/onnxruntime.cmake, cmake/onnxruntime_providers_webgpu.cmake).
Why: the WebGPU EP pulls in user32.dll; delay-loading lets onnxruntime.dll load in processes with Win32k lockdown, where user32.dll is never actually needed because device discovery is skipped.

Known limitation (optimization scope)

In this initial version, only basic-level graph transformations (default constant-folding/L1 passes plus layout transformation) are offloaded to the compile-only session. WebGPU EP-specific and higher-level fusions (ORT optimizer levels L2–L4) are not yet captured in the produced model. This is not specific to the WebGPU EP: it is a pre-existing property of the CompileModel (embed) API, which serializes the intermediate graph at the partitioning boundary — i.e. before the L2–L4 optimizer passes run — so those passes cannot contribute to the serialized output for any EP. The compile-only session actually therefore uses ORT_ENABLE_BASIC optimization level, since any higher level would run L2–L4 without benefit. The longer-term goal is to offload all optimization levels from the compiler process (by adding a post-L4 serialization sink), while the execution process loads the already-optimized model via CreateSessionFromArray with ORT_DISABLE_ALL and thus performs no graph transformation of its own.

Motivation and Context

Since the WebNN Compiler process is sandboxed, graph compilation within this process must be in an offline/device-free mode.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Review details

• Files reviewed: 19/19 changed files
• Comments generated: 2
• Review effort level: Low