Multi-agent RAG research assistant
A supervisor plans each question, three specialist agents answer it, and every routing decision is visible.
- Faithfulness
- 0.93
- Cost per question
- $0.0017
- Time to first token
- −29%
Overview
The assistant answers questions about 94 arXiv papers on LLM agents. A LangGraph supervisor plans each question and delegates it to Research, Data (SQL) and Summary agents, each with its own tools and prompt.
The retrieval pipeline is built end to end: section-aware PDF chunking with PyMuPDF produces 2,701 chunks, embeddings land in Chroma, and every citation links back to the passage it came from. Live web search arrives through Tavily's remote MCP server, with one MCP tool shared by two agents.
Challenges
Retrieval missed on questions whose answers sat in one particular section. Prepending the paper title and section to each embedded chunk raised top-1 retrieval from 10/12 to 12/12 in spot checks.
Neither provider won everywhere, so every model call routes between OpenAI GPT 6 Luna and Anthropic Claude Haiku 5.5 using rules from a head-to-head bake-off, with automatic failover and the reason for each routing decision surfaced in the UI.
Time to first token was dominated by each agent's first tool call. Pre-fetching it cut TTFT by 29% with no measurable loss in answer quality.
Decisions
The calls that shaped this project, the alternatives they beat, and what each one cost.
Embed the paper title and section with every chunk
over embed the chunk text alone
WhyA chunk pulled out of a paper loses the context that makes it findable. Carrying its title and section into the embedding raised top-1 retrieval from 10/12 to 12/12.
Trade-offSpends part of each chunk's embedding budget on metadata rather than content.
Route every call between two providers
over pick one model and use it everywhere
WhyThe bake-off showed neither model won on every task. Per-call routing takes the better of the two and gives automatic failover when one provider is down.
Trade-offTwo SDKs and behaviour that varies by request, so the UI shows why each call went where it did.
A supervisor delegating to specialist agents
over one agent holding every tool
WhyA single agent picks tools badly as the tool count grows. Splitting Research, Data and Summary into separate prompts and toolsets keeps each decision small enough to get right.
Trade-offAn extra planning hop on every question.
Pre-fetch each agent's first tool call
over wait for the plan to finish before acting
WhyThe supervisor's plan nearly always opens an agent with the same first tool call, so issuing it speculatively removed 29% of the wait before the first token.
Trade-offOccasional wasted calls when the plan diverges from the guess.
Outcomes
RAGAS on 28 hand-checked questions: 0.93 faithfulness and 0.93 context recall, at roughly $0.0017 per question.
Answers stream token by token over Server-Sent Events from FastAPI to a vanilla JS front end that shows the plan, the model choices, every tool call and the citations.
Deployed on Google Cloud Run with Firestore-backed conversation memory and keys in Secret Manager, behind 79 automated tests that inject provider, tool and worker failures.