vllm-project /semantic-router
An open, programmable decision layer for models and compute.
★ 6,479⑂ 1,122👁 61⑃ 3,930● 2,693▤ 4
+2
日增长
6,479
stars
1,122
forks
61
watchers
3,930
prs
2,693
issues
4
releases
README
An open, programmable decision layer for models and compute. Documentation | System Duo | Blog | Publications | Hugging Face | Slack !GitHub Release !Go --- About Intelligence beyond any one model. Give your agent harness one API for many models. vLLM Semantic Router selects or combines models for each call, guided by your policy. Your harness keeps the agent loop, tools, and task state. The Router chooses among configured backends across local, private, and cloud compute. | Dimension | Fragmented today | With vLLM SR | | --- | --- | --- | | Models | Different models excel at different tasks. | Select or combine models. | | Compute | Hardware varies in speed and capacity. | Choose among configured backends. | | Location | Edge, private, and cloud. | Keep calls within approved locations. | | Preference | Priorities change by task. | Set quality, latency, and cost priorities. | Explore how it works → Getting Started Install bash curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel stable For pip, uv, or agent-driven installation, see the Installation Guide. Try it online Try System Duo: pick an example and it runs live in your browser, with a free key on first run. Latest News - [2026/10/14] d3 ranks 1 on text and vision in the Jev Decision Index, with smaller sizes on the Pareto frontier. - [2026/10/14] Decision 3.0, Decision 2.0, and Vela 2.0 reached Hugging Face Trending Collections. - [2026/10/06] Vela 2.0: Towards Open Foundation Routing Models - [2026/09/24] vLLM Semantic Router v0.4 Hermes: Many Models, One Improving System - [2026/09/22] Introducing Decision 1.0: Open Decision Foundation Models - [2026/09/18] Introducing Vela 1.0 - [2026/08/24] Find Your Focus: How to Join and Work Together - [2026/08/05] LettuceDetect v2 in Semantic Router: Generative Hallucination Detection as a vLLM Endpoint Earlier announcements - [2026/07/21] Beyond a Single Model: Building Mixture-of-Models Systems with vLLM Semantic Router - [2026/07/09] Adding Cursor-Style Auto Model Selection to OpenCode with vLLM Semantic Router - [2026/06/29] Micro-Agent: Beat Frontier Models with Collaboration inside Model API - [2026/06/16] Beyond One Model: Fusion in vLLM Semantic Router - [2026/06/05] vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing - [2026/03/24] Vision Paper Released: The Workload-Router-Pool Architecture for LLM Inference Optimization - [2026/03/10] v0.2 Released: vLLM Semantic Router v0.2 Athena Release - [2026/02/27] White Paper Released: Signal Driven Decision Routing for Mixture-of-Modality Models - [2026/01/05] Iris v0.1 Released: vLLM Semantic Router v0.1 Iris: The First Major Release - [2025/12/16] Collaboration: AMD × vLLM Semantic Router: Building the System Intelligence Together - [2025/12/15] New Blog: Token-Level Truth: Real-Time Hallucination Detection for Production LLMs - [2025/11/19] New Blog: Signal-Decision Driven Architecture: Reshaping Semanti
Top issues
#2966[WG] Router Models & Inference Runtime — charter and Lead/Member self-nomination97 comments · 2026-08-22#15[Community] Join the vLLM Semantic Router Workgroups87 comments · 2025-09-01#2987[WG] Agentic & Context — charter and Lead/Member self-nomination54 comments · 2026-08-23#2967[WG] Data Plane & Networking — charter and Lead/Member self-nomination51 comments · 2026-08-22#2976[Epic] Enable safe cross-model KV-cache reuse50 comments · 2026-08-22#2965[WG] MoM & Routing — charter and Lead/Member self-nomination49 comments · 2026-08-22#2970[WG] Developer Experience & Ecosystem — charter and Lead/Member self-nomination36 comments · 2026-08-22#2969[WG] Evaluation & Quality — charter and Lead/Member self-nomination35 comments · 2026-08-22
Discussions
#4608Proposal: a clarification-loop example for decision models with incomplete input0 comments · 2026-10-05#4294Decision-1.0-Lux-9B is listed on systemonemodels.tech: want to claim the page?0 comments · 2026-09-27#3946semantic-router-counterfactual v0.10 comments · 2026-09-19#2952[RFC] Feature Request: Dedicated Routing Strategy for Long-Horizon Code Tasks (Beyond Multi-Domain Routing)0 comments · 2026-08-20#2752How does this auto routing exactly works2 comments · 2026-08-03#2180[RFC] Semantic cache entry identity: should the key be query-only, or composite (tenant × model × query)?0 comments · 2026-06-14#1862Getting More Involved0 comments · 2026-04-30#1860[RFC] Routing Hint Injection & Plan Binding — Escape Hatches for LLM Routing1 comments · 2026-04-29
Latest release
Release v0.4.0 v0.4.0 2026-09-27Announcement 982 commits · 130 contributors · 107 first-time contributors since Themis. This is a milestone built by a growing community across models, runtime, evaluation, and the experience of putting it all to work. Read the full release blog 🎉 Python bash pip install vllm-sr==0.4.0 New Contributors @xiaoyu-xyz made their first contribution in https://github.com/vllm-project/semantic-router/pull/2061 @wyf027 made their first contribution in https://github.com/vllm-project/semantic-router/pull/2187 @omkar-334 made their first contribution in https://github.com/vllm-