Add vitpose-plus-huge recipe + fix FP16 2GB protobuf limit#1099
Draft
ssss141414 wants to merge 2 commits into
Draft
Add vitpose-plus-huge recipe + fix FP16 2GB protobuf limit#1099ssss141414 wants to merge 2 commits into
ssss141414 wants to merge 2 commits into
Conversation
Add keypoint-detection fp16 recipe for usyd-community/vitpose-plus-huge. The ViT-Huge fp32 export is ~3.4 GB, and ORT's convert_float_to_float16 runs shape inference by default, which serializes the proto and exceeds protobuf's 2 GB message limit (EncodeError). convert_to_fp16 now auto-detects large models (>=90% of 2 GB via ByteSize with a raise-fallback) and passes disable_shape_infer=True, unblocking all >2 GB models. Validated CPU-only: genuine fp16 (FLOAT16=948, FLOAT32=0), build 661.9s, perf P50 746.49ms.
c260d41 to
0a301e9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
usyd-community/vitpose-plus-huge(keypoint-detection) + fix FP16 2 GB protobuf limitAdds a recipe for the ViT-Huge VitPose pose-estimation model and fixes an FP16-conversion
crash that blocks any model whose FP32 graph exceeds protobuf's 2 GB message limit.
What this PR changes
src/winml/modelkit/quant/fp16.pyByteSize() >= ~1.8 GB) to avoid the 2 GB protobuf serialization limit. The FP16 cast itself does not need inferred shapes.examples/recipes/usyd-community_vitpose-plus-huge/cpu/cpu/keypoint-detection_fp16_config.jsonVitPoseForPoseEstimation,pixel_values[1,3,256,192], FP16, COCO-keypoints eval config.Baseline gap (why this PR is needed)
Two things were verified against clean
origin/main(3f5e4683):Task auto-detection already works —
winml build -m usyd-community/vitpose-plus-huge(no recipe) exports + optimizes fine; keypoint-detection is auto-detected. FP32
optimized.onnx.data= 3438.5 MB. So the recipe's job is FP16 + eval config, not task fixing.FP16 conversion crashes on
main— converting that 3.4 GB FP32 graph to FP16 (ORT'sconvert_float_to_float16, shape-inference on by default) fails:With the fix (shape inference disabled for large models) the conversion succeeds and
produces a correct FP16 graph:
model.onnx.data= 1719.1 MB (exactly half of FP32).Verification (this run,
winmlv0.2.0, host = CPU-only)Runtime-verified on the EP available on this host (CPUExecutionProvider). Accelerator EP
buckets are carried over as prior-verified on capable hardware/CI and are not re-run here.
--precision fp16)VitPoseForPoseEstimation(outputheatmaps [1,17,64,48])--ep all, FP16 model)build_script scripts/build_coco_keypoints.py); the dataset build is a heavy download and was not run in this environment.Notes
--precision fp16; the recipequant.mode="fp16"aloneis overridden by CPU auto-precision (→ FP32).
to be high — accelerator EPs are the intended targets.