BerriAI /litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
★ 63,301⑂ 13,301👁 236⑃ 36,392● 14,154▤ 1,573
+46
日增长
+348
周增长
63,301
stars
13,301
forks
236
watchers
36,392
prs
14,154
issues
1,573
releases
README
🚅 LiteLLM
LiteLLM AI Gateway
Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.
LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website
---
What is LiteLLM
LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.
Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization.
Jump to LiteLLM Proxy (LLM Gateway) Docs
Jump to Supported LLM Providers
---
Why LiteLLM
Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:
- Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling
- Drop-in OpenAI compatibility — swap providers without rewriting your code
- Production-ready gateway — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box
- 8ms P95 latency at 1k RPS (benchmarks)
OSS Adopters
Netflix
---
Features
LLMs - Call 100+ LLMs (Python SDK + AI Gateway)
All Supported Endpoints - /chat/completions, /responses, /embeddings, /images, /audio, /batches, /rerank, /a2a, /messages and more.
Python SDK
shell
uv add litellm
An independent litellm-core distribution provides the Python SDK with the same
import litellm API and runtime dependencies. It has no optional extras, CLI entry
points, or bundled dashboard. Install one SDK distribution per environment because
litellm and litellm-core own overlapping Python files. Use litellm for the
proxy, CLI, and optional extras
Build and install core from this checkout while its release integration is pending:
shell
python litellm/scripts/build_core_distribution.py --out-dir dist/core
python -m pip install dist/core/litellm_core-.whl
The builder requires Git, uv and the Rust build toolchain. It reads the release version
from the root pyproject.toml, leaves source files unchanged, and produces a wheel
and a self-contained sdist in dist/core. Run installation in a fresh environment
without litellm
python
from litellm import completion
import os
os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"
OpenAI
response = completion(model="openai/gpt-6.1-sol", messages=[{"role": "user", "content": "Hello!"}])
Anthropic
response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}])
AI Gateway (Proxy Server)
Getting Started - E2E Tutorial - Setup virtual keys,Top issues
#31263LiteLLM Rust Migration - the fastest and litest AI Gateway (sub 1ms overheads)33 comments · 2026-06-25#30484LiteLLM Stability Sprint Roadmap26 comments · 2026-06-15#39057Design intent: on cache hits, spend is zeroed but token columns replay the original usage — which basis should token reports aggregate?26 comments · 2026-09-01#18686I WISH LITELLM HAD - Models, Providers, Endpoints requests24 comments · 2026-01-06#26672[Bug]: Budget enforcement bypassed in v1.82.3 for key/user max_budget despite spend exceeding max_budget21 comments · 2026-04-28#8513[Feature]: Set currency by env variable21 comments · 2025-02-13#25762[Feature]: unlimited SSO Login with Standard Plan20 comments · 2026-04-15#9753Notice: Deprecation of the Prisma Python client20 comments · 2025-04-04
Discussions
#51749[Withdrawn] posted by mistake — please ignore1 comments · 2026-11-13#51436Gemini 4 Hydrogen API Pricing and Day 0 Support on LiteLLM0 comments · 2026-11-12#51333mcp-slim: Cross-provider MCP schema normalizer, token optimizer (-50% to -98%), and secret firewall0 comments · 2026-11-12#51157Capping one agent run in front of a LiteLLM proxy, and sending less each turn0 comments · 2026-11-11#50970Gemini 4 Carbon is now supported on LiteLLM0 comments · 2026-11-10#50654LiteLLM enabling in-depth analysis of agentic behaviours0 comments · 2026-11-08#50640Counting tokens per customer when your API calls models through LiteLLM0 comments · 2026-11-08#50533models not available1 comments · 2026-11-07
Latest release
v1.109.1 v1.109.1 2026-11-12Verify Docker Image Signature All LiteLLM Docker images are signed with cosign. This release is signed with the key introduced in commit e82a3c6. Verify using the pinned commit hash (recommended): A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the published signing key: bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.109.1 Verify using the release tag (convenience): Tags are protected in this repository and resolve to the s
相关报道 (5)
News & coverage (10)
Hacker NewsTell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised▲ 938 · 500 comments · 2026-03-24Hacker NewsMalicious litellm_init.pth in litellm 1.82.8 PyPI package – credential stealer▲ 739 · 1 comments · 2026-03-24Hacker NewsShow HN: liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching▲ 140 · 34 comments · 2023-08-12Hacker NewsShow HN: Litellm – Simple library to standardize OpenAI, Cohere, Azure LLM I/O▲ 62 · 17 comments · 2023-07-27Hacker NewsShow HN: A modern C++20 AI SDK (GPT‑4o, Claude 3.5, tool‑calling)▲ 56 · 6 comments · 2025-06-29Hacker NewsLiteLLM Python SDK Proxy Server LLM Gateway Call 100 LLM APIs in OpenAI Format▲ 5 · 0 comments · 2025-07-24Hacker NewsLitellm 1.107.0 on PYPI Compromised▲ 3 · 0 comments · 2026-10-21Hacker NewsLiteLLM Agent Platform: Run Claude Code/Codex On-Prem Sandboxes and Vaults▲ 3 · 0 comments · 2026-05-16The Record from Recorded Future NewsPopular AI tool LiteLLM says unrevoked key allowed hackers to target users2026-10-26Crypto BriefingLiteLLM traces supply chain attack to a GitHub key nobody revoked2026-10-26