Md Tarek — Portfolio© 2026

Full-stack Developer

System Design — built to hold up.

dhaka → remote, worldwide

Loading experience

Md TarekTarek
AI

Production RAG Architecture: Hybrid Retrieval to Agents

By: Md Tarek · 17 Oct 2026

Abstract visualization of a retrieval-augmented generation pipeline showing embedding search, BM25 fusion, cross-encoder reranking, and agent orchestration layers
Abstract visualization of a retrieval-augmented generation pipeline showing embedding search, BM25 fusion, cross-encoder reranking, and agent orchestration layers

Executive Summary & Key Takeaways

  • Naive RAG is a demo, not a system. The embed → top-k → stuff prompt pattern fails predictably on three classes of query: multi-hop questions, exact-match identifiers (SKUs, error codes, statute numbers, function names), and corpora where user phrasing diverges sharply from document phrasing.
  • Hybrid retrieval is the default. Fusing dense vectors with BM25 through Reciprocal Rank Fusion (RRF) recovers the exact-token recall that bi-encoders systematically lose, at the cost of one extra index and roughly 5–15 ms per query.
  • The reranker is the highest-ROI component in the stack. A cross-encoder over 30–50 fused candidates typically buys more answer quality than swapping embedding models, because bi-encoders compress a 400-token chunk into a single vector *before* they ever see the question.
  • Contextual chunk enrichment beats cleverer chunking. Prepending a short, LLM-generated summary of each chunk's role in its parent document — Anthropic's contextual retrieval technique — was reported to cut failed retrievals by roughly 49%, and by roughly 67% when combined with BM25 and reranking.
  • Agents are a cost multiplier, not a quality guarantee. Add planning and looping only after a single-shot hybrid + rerank baseline is measured and demonstrably insufficient; every additional hop multiplies tokens, latency, and non-determinism.
  • Instrument retrieval separately from generation. Recall@k, nDCG@10, and MRR tell you whether the evidence ever reached the prompt. Faithfulness and answer-relevance scores tell you whether the model used it. Conflating the two makes every regression undebuggable.
  • Design to an explicit latency budget. For interactive chat, target p95 ≤ 2 s end to end: ~10 ms query embedding, 20–60 ms hybrid search, 30–200 ms rerank, remainder on generation with streaming.

The Anatomy of a Production Retrieval Pipeline

Ingestion is where most RAG projects actually die

Retrieval quality is capped by parsing quality. A layout-blind PDF text extractor will shred tables into interleaved cell fragments, orphan headings from their sections, and drop page numbers you need for citations. Use a layout-aware parser (Unstructured, Docling, or PyMuPDF with custom table extraction), preserve tables as Markdown, and carry the heading path (doc > section > subsection) as metadata rather than as body text.

For chunking, start at 256–512 tokens with 10–15% overlap and split recursively on structural boundaries — heading, then paragraph, then sentence — rather than on a fixed token window. Semantic and late chunking are worth benchmarking, but well-executed structural chunking usually captures most of the gain at a fraction of the ingestion cost.

Metadata is not decoration. Source URI, section path, effective date, and tenant/ACL identifiers all become filters. And filtering semantics matter: post-filtering a top-10 ANN result set with a tight tenant filter can leave you with two rows. Either use a payload-indexed filtered search (Qdrant payload indexes, pgvector partial indexes, Weaviate where clauses) or over-fetch by 5–10× before filtering.

Hybrid search and Reciprocal Rank Fusion

Dense retrieval answers *what does this mean*; BM25 answers *does this contain the exact string*. Production systems need both. The standard fusion is rank-based, which sidesteps the fact that cosine similarity (bounded, compressed) and BM25 (unbounded, corpus-dependent) live on incomparable scales:

RRF(d) = Σ_i  w_i / (k + rank_i(d))        with k = 60

A document ranked 1st by dense and 8th by sparse outscores one ranked 3rd by both — rank fusion rewards *agreement across retrievers* without any score normalization. A tuned convex combination (α · dense + (1-α) · sparse) has a higher ceiling but requires per-corpus tuning and re-tuning after any model change, which is why most teams ship RRF first.

On the vector side, pin your embedding model version and honor its instruction prefixes — E5 and BGE models are trained with query: / passage: prefixes, and omitting them is a silent, hard-to-diagnose recall killer. For the index itself, HNSW with M=16–32 and ef_construction=200, with ef_search tuned upward until it meets your recall target, is a sane default up to roughly 10M vectors on pgvector; beyond that, quantized IVF-PQ or a dedicated engine is worth the operational cost.

Reranking and context assembly

A cross-encoder scores (query, chunk) jointly, so it can see term-level interaction that a dot product over pre-computed vectors cannot. Retrieve 30–50 candidates, rerank, keep 4–6. Expect roughly 30–200 ms depending on model size and hardware; hosted options (Cohere Rerank, Voyage rerank) keep you out of the GPU business. Returns diminish sharply past ~100 candidates — reranking junk does not manufacture relevance.

Assembly rules that consistently matter: deduplicate near-identical chunks by content hash, order evidence so the strongest chunks sit at the start and end of the context (models attend unevenly to the middle), attach stable citation IDs at assembly time rather than asking the model to invent them, and keep chunks verbatim. Summarizing evidence before the final answer is where grounding quietly degrades into paraphrase-and-hallucinate.

# hybrid_rag.py — dense + BM25 fusion with cross-encoder reranking
from typing import Dict, List
import numpy as np
from rank_bm25 import BM25Okapi
from sentence_transformers import CrossEncoder, SentenceTransformer

RRF_K = 60

def rrf(rank_lists: List[List[str]], weights: List[float] = None,
        k: int = RRF_K) -> Dict[str, float]:
    weights = weights or [1.0] * len(rank_lists)
    scores: Dict[str, float] = {}
    for w, ranked in zip(weights, rank_lists):
        for rank, doc_id in enumerate(ranked, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + w / (k + rank)
    return dict(sorted(scores.items(), key=lambda kv: kv[1], reverse=True))

class HybridRetriever:
    def __init__(self, docs: List[Dict[str, str]],
                 embed_model: str = 'BAAI/bge-m3',
                 rerank_model: str = 'BAAI/bge-reranker-v2-m3'):
        self.docs = docs
        self.by_id = {d['id']: d for d in docs}
        self.ids = [d['id'] for d in docs]
        self.texts = [d['text'] for d in docs]
        self.embedder = SentenceTransformer(embed_model)
        self.reranker = CrossEncoder(rerank_model)
        self.bm25 = BM25Okapi([t.lower().split() for t in self.texts])
        self.matrix = self.embedder.encode(self.texts, normalize_embeddings=True)

    def search(self, query: str, k: int = 20, rerank_top: int = 5) -> List[Dict]:
        q = self.embedder.encode(query, normalize_embeddings=True)
        dense = [self.ids[i] for i in np.argsort(-(self.matrix @ q))[: k * 3]]

        sparse_scores = self.bm25.get_scores(query.lower().split())
        sparse = [self.ids[i] for i in np.argsort(-sparse_scores)[: k * 3]]

        # weight sparse lower: BM25 dominates on identifiers, not on prose
        fused = list(rrf([dense, sparse], weights=[1.0, 0.6]))[:k]

        ce = self.reranker.predict([(query, self.by_id[i]['text']) for i in fused])
        order = np.argsort(-ce)[:rerank_top]
        return [{'id': fused[i], 'text': self.by_id[fused[i]]['text'],
                 'score': float(ce[i])} for i in order]

From Retrieval to Reasoning: Bounded Agentic RAG

Single-shot retrieval fails when the question cannot be answered from one semantic neighborhood. The four reliable signals are: ambiguous queries (the system cannot tell which index to hit), multi-hop questions (the second fact is only findable after reading the first), heterogeneous tools (SQL plus documents plus web), and verification-critical domains where a second look is cheaper than a wrong answer.

The patterns, in ascending order of cost:

  • Router. One classification call routes the query to the correct index or tool. The cheapest agentic upgrade and usually the first one worth shipping.
  • Multi-query / decomposition. One call generates 3–5 paraphrases or sub-questions, executed in parallel and fused with RRF. Strong win on short, vague user queries.
  • Corrective RAG (CRAG). Grade the retrieved evidence; if confidence is low, rewrite the query or fall back to a broader source.
  • Self-RAG. The model emits reflection signals deciding whether retrieval is needed at all and critiquing whether its own draft is supported.
  • Plan-and-execute (ReWOO). Plan all sub-queries up front and execute them in parallel — one planning call instead of N sequential ReAct turns, and far less loop risk.
  • ReAct. Interleaved reasoning and acting. Most flexible, most expensive, and the most prone to unbounded loops.

Non-negotiables for anything in production: a hard step budget, deduplication by document ID, an explicit termination predicate (evidence sufficiency *or* budget exhaustion), validated structured tool schemas, and per-step trace logging. And treat retrieved text as untrusted input — a document that can set your tool arguments is a prompt-injection vector. Tool arguments come from the planner, never from retrieved content.

Do the cost math before you ship: a single-shot answer is one retrieval plus one generation. A four-hop agent is four retrievals, five model calls, and roughly 4–6× the spend at 3–5× the latency. If the eval delta is a handful of points, ship single-shot.

# agentic_rag.py — bounded plan → retrieve → critique loop
from dataclasses import dataclass, field
from typing import List

@dataclass
class Budget:
    max_steps: int = 6
    max_reretrievals: int = 3
    steps: int = field(default=0)
    reretrievals: int = field(default=0)

def run_agent(question: str, retriever, llm, budget: Budget = None) -> str:
    b = budget or Budget()
    evidence: List[dict] = []
    pending: List[str] = [question]

    while pending and b.steps < b.max_steps:
        query = pending.pop(0)
        b.steps += 1
        evidence.extend(retriever.search(query, k=10, rerank_top=4))

        verdict = llm.critique(question, evidence)
        # -> {'sufficient': bool, 'sub_queries': [...]}
        if verdict['sufficient'] or b.reretrievals >= b.max_reretrievals:
            break

        b.reretrievals += 1
        pending = verdict.get('sub_queries') or []

    unique = list({e['id']: e for e in evidence}.values())
    return llm.answer(question, unique, require_citations=True)

Evaluation, Observability, and the Cost Curve

Build a golden set of 100–300 question / answer / gold-evidence triples, stratified across four buckets: single-hop, multi-hop, exact-match identifier, and unanswerable. That last bucket is the one teams skip and the one that matters most — a RAG system that never abstains is a liability, not a product.

Measure in two layers. Retrieval: recall@k (was the gold chunk in the top k at all?), nDCG@10, MRR, plus an empty-retrieval rate as a leading indicator of indexing regressions. Generation: faithfulness/groundedness, answer relevance, citation precision. RAGAS automates these with an LLM judge, but treat judge scores as *relative* signals — judges exhibit position bias, verbosity bias, and self-preference for their own model family. Calibrate against 50 human labels before you trust a 0.05-point delta.

In production, watch citation click-through, thumbs-down rate, human escalation rate, the fraction of answers emitted without a citation, p95 latency, and cost per query. Cache exact-match prompts freely; be far more careful with semantic caches (embedding similarity ≥ 0.95), which can serve stale answers after a document update. Invalidate on corpus version and partition by tenant.

Pro-Tips & Common Pitfalls

Ship checklist

  • [ ] Recall@20 is measured on a labeled set *before* any chunk-size or embedding-model decision.
  • [ ] Hybrid (dense + BM25) is on by default; RRF weights are documented and versioned.
  • [ ] A cross-encoder reranker sits between fusion and context assembly, over-fetching ≥ 3× the final context size.
  • [ ] Every prompt logs the exact chunk IDs, scores, and index version that produced it.
  • [ ] The system has an explicit abstain path and an eval bucket that tests it.
  • [ ] p95 latency and cost-per-query are dashboarded next to quality, not separately.

Pitfalls that cost the most time

  • Embedding-model drift. Upgrading the model without a full re-index leaves two incompatible vector spaces in one collection; recall degrades quietly and no error is raised. Treat the embedding model version as part of the index schema.
  • Overfitting the prompt to the eval set. If you tune prompts on the same 100 questions you report scores on, the number is fiction. Hold out 30% and rotate it.
  • Reranking too few candidates. Reranking the top 5 from a dense-only search cannot rescue a chunk that was never retrieved. Fix recall first, then precision.
  • Query–document asymmetry. Short user queries against long technical documents benefit disproportionately from HyDE (generate a hypothetical answer, embed *that*) or multi-query expansion — benchmark both before adding an agent.
  • Summarizing evidence pre-answer. Compression before generation invites hallucination; compress with a reranker instead of a summarizer.
  • Treating judge scores as ground truth. An LLM judge is a noisy proxy. Route low-confidence and high-stakes answers to human review rather than to a second automated opinion.