Infrastructure for continually self‑improving agents

★ 7,829⑂ 739👁 512⑃ 1,033● 415▤ 5
PythonApache-2.0Created 2026-08-31reefinfra.ai
下载:v0.1.3v0.1.3 ↓0.1.32026-10-14
+6
日增长
+40
周增长
7,829
stars
739
forks
512
watchers
1,033
prs
415
issues
5
releases

README

Infrastructure for continually self‑improving agents








English | 中文



Reef is the first open-source infrastructure for continual self-improving agents.
It connects agent inference, feedback, learning, and versioned delivery. Use it
to train model weights with Slime and SGLang, or improve an agent's harness, including its prompts, rules, and skills.




🚀 Get started |
🗺️ Roadmap |
🗓️ Office Hours |
📣 Launch post |
💬 Join Discord |
📱 Join WeChat Group




 📰 News

- 2026-11: We are refactoring Reef to a control plane + workers architecture. Some recipes and examples can fail on main until the refactor is complete. If a recipe fails, please open an issue.
- 2026-11: A scenario can split its traffic across its served artifacts 1227, also through Semantic Router 859. The new python harness adapter evolves a Python program 1242.
- 2026-10: Tinker runs as a training component 901. Episodes and agents can run in Modal Sandboxes 942. The artifact repository can live on Hugging Face 1003. New recipe: OPD 683.
- 2026-09: reef-infra v0.1.0 is on PyPI. New recipes: GEPA, SDFT, SDPO and Meta-Harness. Reef serves the OpenAI Responses API at /v1/responses 528.
- 2026-08: Reef is open source. Read the launch post.

 🎯 When to use Reef

Use Reef when you want your agent to keep improving simply by learning from how you interact with your agent.

| Your goal | Learning path | What you need |
|---|---|---|
| Keep getting stronger model designed for you | Model weight training | A trainable model, a supported GPU stack, and feedback your recipe can use |
| Get your harness to self-improve | Harness optimization | A model endpoint, representative tasks, and an evaluator; no local training GPUs |
| Scientific discoveries | Test-time training | An execution environment, a correctness checker, and a measurable objective |


 🧩 How Reef fits your stack

| Ability | Inference engine (vLLM, SGLang, …) | RL training framework (Slime, veRL, AReaL, …) | Reef |
|---|:---:|:---:|:---:|
| Serves live traffic | ✅ | ❌ | ✅ |
| Trains weights | ❌ | ✅ | ✅ |
| Version management | ❌ | ❌ | ✅ |
| Stays live through updates | ❌ | ❌ | ✅ |
| Evolves beyond weights (skills, harness) | ❌ | ❌ | ✅ |


 🔄 How it works



  
  



Reef processes each learning cycle in four steps. The table also shows which
modules implement each step.

| Step | What happens | Where it lives |
|---|---|---|
| 1 · Serve | Serve agent requests and record interactions. | service/ — agent requests and interaction recordsinference/ — inference and artifact updates |
| 2 · Observe | Match feedback to recorded interactions. | storage/records.py — stored interactions and feedbackrecord2sample/ — feedback matching, eligibility, and samples |
| 3 · Grow | Produce an update from eligible records. | recipe/ — recipe integrationtrain/ — the training pipeline, job data, and training jobs |
| 4 · Commit | Apply the configured selection policy and publish accepted updates. | evaluat

Top issues

Discussions

Latest release

v0.1.3 v0.1.3 2026-10-14
Highlights

- Split training data preparation into record-to-record, record-to-task, and task-to-batch stages (766).
- Add OPD recipes, generated-task training, and AutoEnvScaling examples (683, 707, 755, 790, 791).
- Let recipes choose the training method for each job (695).
- Support Slime weight publication to vLLM through the disk transport (710).
- Share sandbox execution through Harbor environments (803).
- Fix interrupted training recovery, engine startup port conflicts, relative artifact work directories, and OpenRouter web-search streams (728, 730, 723, 712, 769).

 Upgrade notes

Thi

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON