xlite-dev /Awesome-LLM-Inference
📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉
★ 5,558⑂ 447👁 132⑃ 187● 13▤ 45
+1
日增长
5,558
stars
447
forks
132
watchers
187
prs
13
issues
45
releases
README
📒Introduction
Awesome-LLM-Inference: A curated list of 📙Awesome LLM Inference Papers with Codes. For Awesome Diffusion Inference, please check 📖Awesome-DiT-Inference !. For CUDA learn notes, please check 📖LeetCUDA !.
📖 News 🔥🔥
- [2026/03] Cache-DiT 🎉v1.3.0 release is ready, the major updates including: Ring Attention w/ batched P2P, USP (Hybrid Ring and Ulysses), Hybrid 2D and 3D Parallelism (💥USP + TP), VAE-P Comm overhead reduce.
!arch
©️Citations
BibTeX
@misc{Awesome-LLM-Inference@2024,
title={Awesome-LLM-Inference: A curated list of Awesome LLM Inference Papers with codes},
url={https://github.com/xlite-dev/Awesome-LLM-Inference},
note={Open-source software available at https://github.com/xlite-dev/Awesome-LLM-Inference},
author={xlite-dev, liyucheng09 etc},
year={2024}
}
🎉Awesome LLM Inference Papers with Codes
Awesome LLM Inference for Beginners.pdf: 500 pages, FastServe, FlashAttention 1/2, FlexGen, FP8, LLM.int8(), PagedAttention, RoPE, SmoothQuant, WINT8/4, Continuous Batching, ZeroQuant 1/2/FP, AWQ etc.
🎉Download All PDFs
bash
python3 download_pdfs.py The code is generated by Doubao AI
📖Contents
📖Trending LLM/VLM Topics🔥🔥🔥
📖DeepSeek/MLA Topics🔥🔥🔥
📖Multi-GPUs/Multi-Nodes Parallelism🔥🔥🔥
📖Disaggregating Prefill and Decoding🔥🔥🔥
📖LLM Algorithmic/Eval Survey
📖LLM Train/Inference Framework/Design
📖Weight/Activation Quantize/Compress🔥
📖Continuous/In-flight Batching
📖IO/FLOPs-Aware/Sparse Attention🔥
📖KV Cache Scheduling/Quantize/Dropping🔥
📖Prompt/Context Compression🔥
📖Long Context Attention/KV Cache Optimization🔥🔥
📖Early-Exit/Intermediate Layer Decoding
📖Parallel Decoding/Sampling🔥
📖Structured Prune/KD/Weight Sparse
📖Mixture-of-Experts(MoE) LLM Inference🔥
📖CPU/NPU/FPGA/Mobile Inference
📖Non Transformer Architecture🔥
📖GEMM/Tensor Cores/WMMA/Parallel
📖VLM/Position Embed/Others
📖LLM Inference Applications
📖Trending LLM/VLM Topics (©️back👆🏻)
|Date|Title|Paper|Code|Recom|
|:---:|:---:|:---:|:---:|:---:|
| 2026.03 | 🔥🔥🔥[OneComp] OneComp: One-Line Revolution for Generative AI Model Compression(@Fujitsu) | [[pdf]](https://arxiv.org/pdf/2603.28845) | [[OneCompression]](https://github.com/FujitsuResearch/OneCompression) ! | ⭐️⭐️ |
| 2025.12 | 🔥🔥[QEP] QEP: Quantization Error Propagation, NeurIPS 2025(@Fujitsu) | [[pdf]](https://openreview.net/pdf?id=a3l3K9khbL) | [[OneCompression]](https://github.com/FujitsuResearch/OneCompression) ! | ⭐️⭐️ |
|2024.04| 🔥🔥🔥[Open-Sora] Open-Sora: Democratizing Efficient Video Production for All(@hpcaitech)|[[docs]](https://github.com/hpcaitech/Open-Sora/blob/main/docs/zh_CN/README.md) | [[Open-Sora]](https://github.com/hpcaitech/Open-Sora) !| ⭐️⭐️ |
|2024.04| 🔥🔥🔥[Open-Sora Plan] Open-Sora Plan: This project aim to reproduce Sora (Open AI T2V model)(@PKU)|[[report]](https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/main/docs/Report-v1.0.0.md) | [[Open-Sora-Plan]](https://github.com/PKU-YuanGroup/Open-Sora-Plan) !| ⭐️⭐️ |
|2024.05| 🔥🔥🔥[DeepSeek-V2] DeepSeek-V2: Top issues
Latest release
v2.6.20 v2.6.20 2025-06-17What's Changed Add 4 papers by @woominsong in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/148 Update new paper (KVzip) by @Janghyun1230 in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/149 Add a new paper (GuidedQuant) by @jusjinuk in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/150 Add STAND by @woominsong in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/151 Add Inference-Time Hyper-Scaling by @CStanKonrad in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/152 New Contributors @woominsong made their first contribution in https