NVIDIA /cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

★ 1,000⑂ 338👁 18⑃ 2,173● 441▤ 111
CudaApache-2.0Created 2021-01-22docs.nvidia.com/deeplearning/cudnn/latest/
下载:v1.32.3 releasev1.32.3 ↓1.32.32026-11-10
+0
日增长
1,000
stars
338
forks
18
watchers
2,173
prs
441
issues
111
releases

README

cuDNN Frontend (FE)







cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels — scaled dot-product attention (SDPA / Flash Attention), grouped GEMM fusions for Mixture-of-Experts (MoE) training, fused normalization + activation, and more.

It provides a header-only C++ API and a Python interface (with native PyTorch integration) to the cuDNN Graph API, targeting NVIDIA Hopper (H100/H200) and Blackwell (B200/GB200/GB300) GPUs across FP16, BF16, FP8, and MXFP8 precision.

Links: Documentation · Blog & Deep Dives · PyPI · Release Notes · Samples

 🚀 Latest news:

We will begin open-sourcing kernels based on customer needs, with the goal to educate developers and enable them to customize as needed.

We are now shipping OSS kernels, allowing you to inspect, modify, and contribute to the core logic. Check out our latest implementations:

   FROST GEMM engine: JIT-compiled Blackwell GEMM engine reachable through the ordinary cudnn.pygraph API — matmul, grouped (MoE) matmul, block-scaled FP4/FP8, and chained pointwise epilogues are fused into one kernel from the graph you already built. Opt in with CUDNN_FRONTEND_ENABLE_FROST_ENGINES=1; it is then a candidate for every matmul graph it can serve, ranked against the backend's own plans.
   GEMM + Amax: Optimized FP8 matrix multiplication with absolute maximum calculation.
   GEMM + SwiGLU: High-performance implementation of the SwiGLU activation fused with GEMM.
   GEMM + sReLU: High-performance implementation of squared-ReLU fused with GEMM.
   GEMM + dsReLU: High-performance implementation of dsquared-ReLU fused with GEMM.
   Grouped GEMM (BF16): Unfused BF16 grouped GEMM with dense and discrete MoE weight layouts.
   Grouped GEMM + GLU: Unified BF16 and legacy block-scaled grouped GEMM GLU API supporting dense and discrete MoE weight layouts.
   Grouped GEMM + GLU + Hadamard: Dense grouped GEMM GLU forward fusion with a fused Hadamard transform and per-expert AMAX reduction.
   Grouped GEMM + dGLU: Unified BF16 and legacy block-scaled grouped GEMM dGLU backward API supporting dense and discrete MoE weight layouts.
   Grouped GEMM + SwiGLU: SwiGLU activation fused with Grouped GEMM.
   Grouped GEMM + dSwiglu: dSwiglu activation fused with Grouped GEMM.
   Grouped GEMM + sReLU: Contiguous grouped squared-ReLU GEMM for MoE workloads.
   Grouped GEMM + dsReLU: Contiguous grouped dsquared-ReLU GEMM for MoE workloads.
   Discrete Grouped GEMM + SwiGLU: Per-expert-pointer SwiGLU grouped GEMM for MoE workloads without weight packing.
   Discrete Grouped GEMM + dSwiGLU: Per-expert-pointer dSwiGLU backward grouped GEMM for MoE workloads without weight packing.
   Grouped GEMM + Quant: Legacy dense-only grouped GEMM quant API for MoE FC2/dFC1 workloads.
   Grouped GEMM + Quant (Unified): Unified grouped GEMM quant API with per-row gating for MoE FC2/dFC1 workloads.
   Grouped GEMM + Wgrad: Unified BF16 and legacy block-sc

Top issues

Discussions

Latest release

v1.32.3 release v1.32.3 2026-11-10
- DSA backward: Improve deterministic dKV relay performance on SM100 and add relay cluster-size selection (2072, 2082).
- DSA backward: Add deterministic execution on SM90 with configurable relay cluster sizes (2209).
- DSA indexer: Add deterministic backward execution on SM90 and improve dW reduction for top-k values of 512 and above (2249).
- CSA compressor: Enable compression ratios 2 and 4 on SM90 and tune ratio-2 backward schedules (2247).
- PyTorch SDPA: Detect the cuDNN SDPA switch in torch.nn.attention to avoid duplicate provider registration (2368).

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON