The retriever can return irrelevant documents. Important information may be buried in the middle of a document. A query may be too vague for semantic search. Retrieved chunks may lose their surrounding context. And sometimes the model does not need retrieval at all.
The quality of a RAG system therefore depends heavily on how information is retrieved, filtered, ranked, compressed, and presented to the model.
This guide covers nine techniques that address different parts of the RAG pipeline: Reranking Hybrid Search Chunking Strategies Multi-Query Retrieval Parent Document Retrieval Context Compression HyDE Self-RAG CRAG Reranking
A vector database may search through hundreds or thousands of documents and return the top 20 candidate chunks.
If the application only sends the top 3–5 chunks to the LLM, the correct information never reaches the model.
Without reranking, the most relevant chunk might be ranked #19 and never reach the model.
With reranking, a reranker evaluates the retrieved candidates using both the query and the content of each chunk, promoting the most relevant results to the top.
The reranker can move a previously low-ranked but highly relevant chunk to the top.
Unlike basic vector similarity, a reranker can inspect the relationship between the entire query and the retrieved document.
This helps it: Understand context better Find hidden relevance Filter out noise Improve the quality of the final answer
Common approaches include: Cross-Encoder High accuracy Slower Bi-Encoder + Rerank Model Balanced performance LLM-based Reranker Potentially highest quality More expensive
A basic vector search might initially return chunks about HTTP status codes, API authentication, or general Node.js syntax.
A reranker can compare each candidate directly against the question and prioritize content discussing connection handling, asynchronous I/O, event loops, connection pooling, and horizontal scaling.
Retrieve more. Rerank intelligently. Let the LLM see the best context, not merely the first context. Hybrid Search
A semantic search might return: Diagnosing cache connection failures Distributed cache troubleshooting Network latency in application infrastructure
These documents may be semantically related, but the exact phrase Redis connection timeout might not appear.
A keyword search such as BM25 can find: Redis connection timeout configuration Fixing Redis client timeout errors Redis socket timeout settings
Popular approaches include: Reciprocal Rank Fusion (RRF) Weighted score combination Relative score fusion Rank-based fusion