rllm-org /rllm

Democratizing Reinforcement Learning for LLMs

★ 5,919⑂ 643👁 29⑃ 687● 272▤ 4
PythonApache-2.0Created 2025-01-26docs.rllm-project.com
下载:rLLM: v0.3.0-prev0.3.0-pre ↓v0.3.0-pre2026-04-30
+1
日增长
5,919
stars
643
forks
29
watchers
687
prs
272
issues
4
releases

README

Agentic RL on any harness, with any backend, on any benchmark.











rLLM is an open-source framework for training language agents with reinforcement learning. Bring any harness, run it in any sandbox, and switch training backends with one flag — the same agent code drives both eval and training.

 Core features

- Any harness. 10+ CLI harnesses (Claude Code, Codex, Terminus-2, mini-swe-agent, opencode, ...) plus Harbor-compatible task dirs. Or wrap your own agent — LangGraph, OpenAI Agents SDK, openai.OpenAI — with @rllm.rollout.
- Any sandbox. Docker, Daytona, Modal, or local — with snapshot + warm-pool acceleration to keep rollouts cheap at training-scale.
- Multiple training backends, one API. verl (distributed multi-GPU), tinker (single-machine), fireworks (Fireworks platform). Switch with one flag.
- 60+ integrated benchmarks. Math, code, MCQ, QA, search, VLM, translation, agentic — Terminal-Bench 2.0, SWE-bench, SkillsBench, AIME, MATH-500, GPQA, and more. rllm eval  auto-pulls and runs.
- Multiple training methods. GRPO, REINFORCE, RLOO, SFT, on-policy distillation, and more.
- Battle-tested. State-of-the-art open-source results (DeepScaleR-1.5B, DeepCoder-14B, DeepSWE-32B, FinQA-4B). Adopted by academic labs and industry research teams (see Community Projects below).

Read more on our documentation site.

 Installation

rLLM requires Python = 3.11. You can install it either directly via pip or build from source.

bash
uv pip install "rllm @ git+https://github.com/rllm-org/rllm.git"


This installs dependencies for running rllm CLI with the tinker backend (single-machine, Tinker API). For other backends:

bash
 Distributed multi-GPU training (verl + vLLM/SGLang)
uv pip install "rllm[verl] @ git+https://github.com/rllm-org/rllm.git"

 Fireworks training platform
uv pip install "rllm[fireworks] @ git+https://github.com/rllm-org/rllm.git"


For building from source or Docker, see the installation guide.

 Quickstart

 Option A: CLI (no code needed)

bash
 1. Configure your model provider
rllm model setup

 2. Evaluate on a benchmark
rllm eval gsm8k

 3. Train with RL
rllm train gsm8k


 Option B: Python API

Define a rollout (your agent) and an evaluator (your reward function), then hand them to the trainer:

python
 my_flow.py
from openai import OpenAI
import rllm
from rllm.types import AgentConfig, Episode, Task, Trajectory

@rllm.rollout
def solve(task: Task, config: AgentConfig) - Episode:
    client = OpenAI(base_url=config.base_url, api_key="EMPTY")
    response = client.chat.completions.create(
        model=config.model,
        messages=[{"role": "user", "content": task.instruction}],
    )
    answer = response.choices[0].message.content or ""
    return Episode(
        trajectories=[Trajectory(name="solver", steps=[])],
        artifacts={"answer": answer},
    )


python
 my_evaluator.py
import rllm
from rllm.eval.types import EvalOutput, Signal
from rllm.types import Episode

@rllm.evaluator
def score(task: dict, episode: 

Top issues

Discussions

Latest release

rLLM: v0.3.0-pre v0.3.0-pre 2026-04-30
What's Changed
 [feature] add support for on policy distillation by @kylemontgomery1 in https://github.com/rllm-org/rllm/pull/356
 [Bugfix] Make sure the tracking logger is now explicitly finished by trainers by @listar2000 in https://github.com/rllm-org/rllm/pull/358
 Add Geo3K Tinker training example with VLM support by @BrianChen26 in https://github.com/rllm-org/rllm/pull/357
 Support custom metric for SDK by @thwu1 in https://github.com/rllm-org/rllm/pull/362
 fix: typo in bibtex by @tongyx361 in https://github.com/rllm-org/rllm/pull/363
 Fireworks API returns raw token ids to avoid 

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON