Add nvfp4 attention support for vLLM serving

[kaix-nv]

What does this PR do?

Type of change: ?

New feature.

• Applies NVFP4 fake quantization to Q/K/P/V.
  • Q uses dynamic scaling and an FP32 QDQ carrier.
  • K/V use global scale 1.0.
  • P uses amax 1.0, rounds to the model dtype before packing, and retains FP32 QDQ accumulation.
• Adds paged prefill and a dedicated split-K decode kernel for long-context serving.
• Avoids V-cache re-quantization:
  • Complete 16-token groups are finalized once.
  • The incomplete group remains pristine in FP16/BF16 and is QDQ on read.
• Validates the complete attention plan before modifying the model and fails loudly for unsupported configurations.

Usage

 cd examples/vllm_serve

 python vllm_serve_sparse_attn.py <MODEL_PATH> -tp 8 \
   --attention-backend FLASH_ATTN \
   --no-enable-prefix-caching \
   --worker-cls quant_sparse_attn_worker.QuantSparseAttnWorker

 ### Testing

 Focused B200 kernel tests:

 PYTEST_VERSION=1 PYTHONPATH=$PWD python -m pytest -q \
   tests/gpu/torch/kernels/common/attention/test_triton_fa_p_qdq.py \
   tests/gpu/torch/kernels/common/attention/test_decode_attention.py \
   tests/gpu/torch/kernels/common/attention/test_triton_fa_paged.py

 Result: 47 passed.

 Focused vLLM integration tests:

 PYTHONPATH=$PWD python -m pytest -q \
   tests/gpu_vllm/torch/sparsity/attention_sparsity/test_sparse_attn_worker.py \
   tests/gpu_vllm/torch/sparsity/attention_sparsity/test_quant_sparse_attn_worker.py

Testing

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

• Is this change backward compatible?: ✅ / ❌ / N/A
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: ✅ / ❌ / N/A
• Did you write any new necessary tests?: ✅ / ❌ / N/A
• Did you update Changelog?: ✅ / ❌ / N/A
• Did you get Claude approval on this PR?: ✅ / ❌ / N/A

Additional Information

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e0503d1f-6c5d-431e-b7d3-28f99935ff24

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch kaix/sparse_attn_quant_compact

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-1898/
Built to branch gh-pages at 2026-07-04 05:06 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[codecov]

Codecov Report

❌ Patch coverage is 1.50943% with 261 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.92%. Comparing base (75b5803) to head (46744e1).

Files with missing lines	Patch %	Lines
...torch/kernels/common/attention/decode_attention.py	0.00%	118 Missing ⚠️
...delopt/torch/kernels/common/attention/triton_fa.py	9.09%	40 Missing ⚠️
...lopt/torch/kernels/quantization/attention/v_qdq.py	0.00%	40 Missing ⚠️
.../torch/sparsity/attention_sparsity/plugins/vllm.py	0.00%	40 Missing ⚠️
modelopt/torch/quantization/plugins/vllm.py	0.00%	19 Missing ⚠️
...t/torch/kernels/quantization/common/nvfp4_quant.py	0.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1898      +/-   ##
==========================================
- Coverage   61.21%   60.92%   -0.30%     
==========================================
  Files         515      517       +2     
  Lines       57245    57477     +232     
==========================================
- Hits        35043    35017      -26     
- Misses      22202    22460     +258     

Flag	Coverage Δ
unit	54.69% <1.50%> (-0.24%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.