PeterGriffinJin /Search-R1

Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL

★ 5,579⑂ 503👁 17⑃ 27● 170▤ 0
PythonApache-2.0Created 2025-02-28arxiv.org/pdf/2503.09516
+0
日增长
5,579
stars
503
forks
17
watchers
27
prs
170
issues
0
releases

README

Search-R1: Train your LLMs to reason and call a search engine with reinforcement learning


  



  
    
  
  
    
  
  
    
  
  
    
  
  
    
  



Search-R1 is a reinforcement learning framework for training reasoning and searching (tool-call) interleaved LLMs.  --

Search-R1 is a reinforcement learning framework designed for training reasoning-and-searching interleaved LLMs—language models that learn to reason and make tool calls (e.g., to search engines) in a coordinated manner.

DeepSeek-R1(-Zero) with interleaved search engine calling and an opensource RL training-based solution for OpenAI DeepResearch. --
Built upon veRL, Search-R1 extends the ideas of DeepSeek-R1(-Zero) by incorporating interleaved search engine access and provides a fully open-source RL training pipeline. It serves as an alternative and open solution to OpenAI DeepResearch, enabling research and development in tool-augmented LLM reasoning.



We support different RL methods (e.g., PPO, GRPO, reinforce), different LLMs (e.g., llama3, Qwen2.5, etc) and different search engines (e.g., local sparse/dense retrievers and online search engines).

Paper: link1, link2; Model and data: link; Twitter thread: link; Full experiment log: prelim; v0.1; v0.2; v0.3. Details about these logs and methods can be find here.


!single-turn

 News

- [2025.10] Search-R1 is featured by Thinking Machines Lab's first product Tinker! Details: Document.
- [2025.7] Search-R1 is supported by SkyRL! Detailed instructions: code, Document.
- [2025.6] Search-R1 is now integrated into the latest version of veRL and can take advantage of its most up-to-date features! Detailed instructions: veRL, English Document, Chinese Document.
- [2025.5] The second paper conducting detailed empirical studies is published with logs: v0.3. 
- [2025.4] We support multinode training for 30B+ LLMs!
- [2025.4] We support different search engines including sparse local retriever, dense local retriever with ANN indexing and online search engines!
- [2025.3] The first Search-R1 paper is published with the logs: v0.1; v0.2.
- [2025.2] We opensource Search-R1 codebase with preliminary results.

 Links

- Installation
- Quick start
- Preliminary results
- Inference
- Use your own dataset
- Use your own search engine
- Features
- Ackowledge
- Citations

 Installation

 Search-r1 environment
bash
conda create -n searchr1 python=3.9
conda activate searchr1
 install torch [or you can skip this step and let vllm to install the correct version for you]
pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu121
 install vllm
pip3 install vllm==0.6.3  or you can install 0.5.4, 0.4.2 and 0.3.1

 verl
pip install -e .

 flash attention 2
pip3 install flash-attn --no-build-isolation
pip install wandb


 Retriever environment (optional)
If you would like to call a local retriever as the search engine, you can install the environment as follows. (We recommend using a seperate environment.)
bash
conda create -n retriever

Top issues

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON