Hybrid RAG Configuration
Recommended settings for deploying a Hybrid RAG system with vector + BM25 search.
Hybrid RAG combines vector (embedding) search with BM25 keyword matching for improved retrieval accuracy. This approach captures both semantic meaning and exact keyword matches.
Configuration Flow
Recommended Settings
The following settings balance semantic search with keyword matching. Tune the BM25 weight based on your evaluation results to find the optimal mix for your domain.
Step 1: Project Setup
Project setup defines the identity and purpose of your RAG system. Hybrid RAG works well for domains with mixed query patterns — both natural language questions and exact keyword searches.
| Setting | Value |
|---|---|
| Domain | Any domain with mixed query patterns |
| Use Case | Document Search or Research Assistant |
Step 2: Knowledge Sources
Add your documents through direct upload or from your Document Library. Hybrid RAG handles larger and more diverse document collections well, making it suitable for multi-domain knowledge bases.
| Setting | Value |
|---|---|
| Source | Document Upload or Document Library |
| Max File Size | 50 MB per file |
| Recommended Formats | PDF, Markdown, DOCX, HTML |
Step 3: Document Processing
Document processing prepares your raw text for both embedding and keyword indexing. Hybrid RAG benefits from more sophisticated chunking strategies that preserve document structure while maintaining semantic coherence.
| Setting | Value |
|---|---|
| Chunking Method | Recursive or Semantic |
| Chunk Size | 500 tokens |
| Overlap | 10-15% |
| Respect Sentence Boundaries | Enabled |
Step 5: Pipeline Configuration
Pipeline configuration controls how queries are processed and results are returned. Hybrid RAG combines vector and BM25 search results using Reciprocal Rank Fusion for optimal ranking.
| Setting | Value |
|---|---|
| Search Method | Hybrid |
| Embedding Model | all-mpnet-base-v2 or BGE-Large-EN-v1.5 |
| Similarity Function | Cosine |
| Retrieval Method | Custom Document Template or Contextual Retrieval |
| BM25 | Enabled |
| BM25 Weight | 0.3 |
| Rerank Method | RRF (Reciprocal Rank Fusion) |
| LLM Integration | Optional |
| Top K | 5 |
| Score Threshold | 0.7 |
Step 6: API Setup
Once configured, generate your API key and integrate using REST API, Python SDK, or MCP Protocol. Test your endpoints before deploying to production.
| Setting | Value |
|---|---|
| API Key | Generated automatically (starts with rk_) |
| Integration | REST API, Python SDK, or MCP |
| Test Endpoints | Use built-in testing tools |
How Hybrid Search Works
- Vector Search: Embeds the query and finds semantically similar chunks
- BM25 Search: Matches exact keywords and phrases from the query
- Reranking: Combines both result sets using Reciprocal Rank Fusion
- Final Results: Returns the top-K chunks from the merged ranking
BM25 Weight Tuning
| Weight | Effect |
|---|---|
| 0.0-0.2 | Vector-dominant (semantic meaning matters most) |
| 0.3-0.5 | Balanced (recommended starting point) |
| 0.6-0.8 | BM25-dominant (keyword matching matters most) |
| 1.0 | BM25-only (keyword search only) |
When to Use Hybrid RAG
- Users search with both natural language and exact keywords
- Technical documentation with specific product names or codes
- Legal or compliance documents where exact terms matter
- Mixed content with both conceptual and factual information
- When Simple RAG misses keyword-specific queries
When to Upgrade to Graph RAG
Consider Graph RAG when:
- You need to answer relationship-based questions ("Who manages X?")
- Your domain has complex entity relationships
- Users ask multi-hop questions across documents
- You need global summarization of large document sets
Start with BM25 weight at 0.3 and adjust based on your evaluation. If keyword queries are underperforming, increase the weight. If semantic queries suffer, decrease it.