BerriAI /litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

★ 63,301⑂ 13,301👁 236⑃ 36,392● 14,154▤ 1,573
PythonNOASSERTIONCreated 2023-07-27docs.litellm.ai/docs/
下载:v1.109.1v1.109.1 ↓1.109.12026-11-12
+46
日增长
+348
周增长
63,301
stars
13,301
forks
236
watchers
36,392
prs
14,154
issues
1,573
releases

README

🚅 LiteLLM
    
    
        LiteLLM AI Gateway
        
        Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.
        
        
        
        
        
        
    
LiteLLM Proxy Server (AI Gateway) |  Hosted Proxy | Enterprise Tier | Website

    
        
    
    
        
    
    
        
    
    
        
    
    
        
    
    
        
    
    
        
    




---

 What is LiteLLM

LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.

Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization.

Jump to LiteLLM Proxy (LLM Gateway) Docs 
Jump to Supported LLM Providers

---

 Why LiteLLM

Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:

- Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling
- Drop-in OpenAI compatibility — swap providers without rewriting your code
- Production-ready gateway — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box
- 8ms P95 latency at 1k RPS (benchmarks)

 OSS Adopters


  
    
    
    
    
    
    Netflix
    
  


---

 Features


LLMs - Call 100+ LLMs (Python SDK + AI Gateway)

All Supported Endpoints - /chat/completions, /responses, /embeddings, /images, /audio, /batches, /rerank, /a2a, /messages and more.

 Python SDK

shell
uv add litellm


An independent litellm-core distribution provides the Python SDK with the same
import litellm API and runtime dependencies. It has no optional extras, CLI entry
points, or bundled dashboard. Install one SDK distribution per environment because
litellm and litellm-core own overlapping Python files. Use litellm for the
proxy, CLI, and optional extras

Build and install core from this checkout while its release integration is pending:

shell
python litellm/scripts/build_core_distribution.py --out-dir dist/core
python -m pip install dist/core/litellm_core-.whl


The builder requires Git, uv and the Rust build toolchain. It reads the release version
from the root pyproject.toml, leaves source files unchanged, and produces a wheel
and a self-contained sdist in dist/core. Run installation in a fresh environment
without litellm

python
from litellm import completion
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"

 OpenAI
response = completion(model="openai/gpt-6.1-sol", messages=[{"role": "user", "content": "Hello!"}])

 Anthropic  
response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}])


 AI Gateway (Proxy Server)

Getting Started - E2E Tutorial - Setup virtual keys,

Top issues

Discussions

Latest release

v1.109.1 v1.109.1 2026-11-12
Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. This release is signed with the key introduced in commit e82a3c6.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the published signing key:

bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.109.1


Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the s

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON