Usage

Prepare your evaluation data

Tongyi DeepResearch takes a structured question file as input. Two formats are supported.

JSONL (recommended)

Create a file such as eval_data/my_questions.jsonl. Each line is a JSON object with a question field (what you want the model to research) and an answer field (the ground truth used for automatic scoring; leave it empty if you are not benchmarking).

{"question": "What are the latest breakthroughs in quantum error correction?", "answer": ""}
{"question": "Compare the GDP growth of Vietnam and Indonesia from 2015 to 2024.", "answer": ""}
{"question": "What is the capital of France?", "answer": "Paris"}

JSON array

Alternatively, use a single JSON file containing an array of objects:

[
  { "question": "What are the latest breakthroughs in quantum error correction?", "answer": "" },
  { "question": "Compare the GDP growth of Vietnam and Indonesia from 2015 to 2024.", "answer": "" }
]

File attachment support

If you want the model to read a PDF or spreadsheet, place it in eval_data/file_corpus/ and prepend the filename to the question:

{"question": "(Uploaded 1 file: ['report.pdf'])\n\nWhat are the key findings from this report?", "answer": ""}

---

Configure the inference script

Open run_react_infer.sh and set the three required variables at the top:

MODEL_PATH="./model_weights"            # path to downloaded weights OR remote model name
DATASET="eval_data/my_questions.jsonl"  # path to your question file
OUTPUT_PATH="./outputs"                 # directory where results will be saved

The script also has inline comments for every optional API key (web search, page reader, file parser, Python sandbox). Set the ones matching the tools you want to enable.

---

Run inference

bash run_react_infer.sh

The agent will process each question using the ReAct loop: it plans, issues tool calls (web search, page read, Python execution), observes results, and iterates until it produces a final answer. Results are written to OUTPUT_PATH as JSON files, one per question.

---

Using the OpenRouter API path

If you configured Option B (API, no GPU), run the same script after setting MODEL_PATH and the OpenRouter API credentials in inference/react_agent.py. No weight download or GPU is needed.

---

Inference modes

Mode	How to enable	Best for
ReAct	Default in run_react_infer.sh	Benchmarking, evaluating core model ability
IterResearch (Heavy)	Set INFERENCE_MODE=heavy in .env or script	Maximizing answer quality on hard questions

Heavy mode uses test-time scaling (more reasoning iterations) and produces better results at the cost of higher latency and API spend.