dzhng /deep-research
An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine its research direction overtime and deep dive into a topic.
★ 19,906⑂ 2,009👁 109⑃ 66● 107▤ 0
+6
日增长
19,906
stars
2,009
forks
109
watchers
66
prs
107
issues
0
releases
README
Open Deep Research
An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models.
The goal of this repo is to provide the simplest implementation of a deep research agent - e.g. an agent that can refine its research direction over time and deep dive into a topic. Goal is to keep the repo size at
SQ[SERP Queries] --
PR[Process Results]
subgraph Results[Results]
direction TB
NL((Learnings))
ND((Directions))
end
PR -- NL
PR -- ND
DP{depth 0?}
RD["Next Direction:
- Prior Goals
- New Questions
- Learnings"]
MR[Markdown Report]
%% Main Flow
Q & B & D -- DR
%% Results to Decision
NL & ND -- DP
%% Circular Flow
DP --|Yes| RD
RD --|New Context| DR
%% Final Output
DP --|No| MR
%% Styling
classDef input fill:7bed9f,stroke:2ed573,color:black
classDef process fill:70a1ff,stroke:1e90ff,color:black
classDef recursive fill:ffa502,stroke:ff7f50,color:black
classDef output fill:ff4757,stroke:ff6b81,color:black
classDef results fill:a8e6cf,stroke:3b7a57,color:black
class Q,B,D input
class DR,SQ,PR process
class DP,RD recursive
class MR output
class NL,ND results
Features
- Iterative Research: Performs deep research by iteratively generating search queries, processing results, and diving deeper based on findings
- Intelligent Query Generation: Uses LLMs to generate targeted search queries based on research goals and previous findings
- Depth & Breadth Control: Configurable parameters to control how wide (breadth) and deep (depth) the research goes
- Smart Follow-up: Generates follow-up questions to better understand research needs
- Comprehensive Reports: Produces detailed markdown reports with findings and sources
- Concurrent Processing: Handles multiple searches and result processing in parallel for efficiency
Requirements
- Node.js environment
- API keys for:
- Firecrawl API (for web search and content extraction)
- OpenAI API (for o3 mini model)
Setup
Node.js
1. Clone the repository
2. Install dependencies:
bash
npm install
3. Set up environment variables in a .env.local file:
bash
FIRECRAWL_KEY="your_firecrawl_key"
If you want to use your self-hosted Firecrawl, add the following below:
FIRECRAWL_BASE_URL="http://localhost:3002"
OPENAI_KEY="your_openai_key"
To use local LLM, comment out OPENAI_KEY and instead uncomment OPENAI_ENDPOINT and OPENAI_MODEL:
- Set OPENAI_ENDPOINT to the address of your local server (eg."http://localhost:1234/v1")
- Set OPENAI_MODEL to the name of the model loaded in your local server.
Docker
1. Clone the repository
2. Rename .env.example to .env.local and set your API keys
3. Run docker build -f Dockerfile
4. Run the Docker image:
bash
docker compose up -d
5. Execute npm run docker in the docker service:
bash
docker exec -it deep-research npm ruTop issues
#76using with local ollama and search query11 comments · 2025-02-12#52Could not connect to local LLM11 comments · 2025-02-09#65'The model `o3-mini` does not exist or you do not have access to it.'11 comments · 2025-02-11#12[Feature] Fully local integration11 comments · 2025-02-07#42Firecrawl self-hosted has a /search endpoint?9 comments · 2025-02-09#67DeepSeek R1 - JSON issue8 comments · 2025-02-11#55Vercel button8 comments · 2025-02-09#25Use for Other LLM's API key8 comments · 2025-02-07