RAG Pipeline with Hybrid Search
Production RAG over 500K+ enterprise docs: hybrid dense+BM25 retrieval, cross-encoder reranking, verified citations, and confidence scoring, fully offline on Ollama.
Most RAG demos index one PDF and stop. Production needs more.
The questions that decide whether a RAG system is trustworthy are never about indexing a single document, they are about what happens when retrieval is ambiguous, when a citation does not actually support a claim, and when the answer simply is not in the corpus.
I built this on the EnterpriseRAG-Bench dataset, 500K+ documents spanning Confluence, GitHub, Slack, Gmail, and Jira, with 500 golden Q&A pairs, specifically to stress the production concerns demos skip.
Hybrid retrieval, then verify everything before answering.
Dense vector search catches semantic meaning; BM25 sparse search catches exact keywords, function names, and config keys. Reciprocal Rank Fusion combines both with configurable weighting. A cross-encoder then reranks the top 20 candidates down to the 5 most relevant.
Every inline citation in the generated answer is verified against its source by an LLM judge, and a three-dimensional confidence score (retrieval relevance, citation coverage, answer completeness) decides whether to answer or return a structured "I don’t know" with partial findings instead of hallucinating.
Correctness first, every layer is tunable for speed.
The full pipeline runs ~60, 80s per query on a MacBook with Ollama because it reranks, verifies citations, and scores confidence before responding. That tradeoff is deliberate: for compliance, legal, or medical docs, a wrong answer is worse than a slow one.
Three chunking strategies (fixed, recursive-by-header, semantic) are switchable per-ingestion with side-by-side evaluation, and near-duplicate detection (cosine > 0.95) stops the retriever wasting context-window slots.
Fully offline, fully tested, and honest about uncertainty.
No API keys, no cloud dependency, everything runs locally via Ollama. 59 automated tests cover ingestion, fusion logic, citation parsing, confidence scoring, and OpenAPI schema correctness, and the system reached 85.5% (hybrid) and 92.4% (dense-only) confidence on grounded answers while gracefully declining out-of-scope questions.