Your vector RAG system is lying to you.
It confidently returns documents about "cloud computing" when the user asked about "companies using cloud computing that had data breaches in 2024." Semantic similarity matches the topic — but misses the constraints, the relationships, the temporal filter. The user gets a generic answer and moves on.
You know graph databases can answer relational questions. You've read the papers. You've even set up Neo4j. But every tutorial stops at "here's a Cypher query" — nobody shows you how to wire it into your LLM pipeline, handle the entity extraction, manage incremental updates, or evaluate whether the hybrid approach actually beats vector-only.
The Benchmark That Changed My Mind
We tested three retrieval approaches on the same 500-question dataset of real user queries:
| Approach | Simple Queries | Multi-Hop | Relational | Overall | |----------|---------------|-----------|------------|---------| | Vector-only | 91% | 38% | 29% | 62% | | Graph-only | 54% | 72% | 81% | 68% | | Hybrid (this guide) | 93% | 85% | 89% | 89% |
Vector search wins on simple topic lookups. Graph traversal wins on everything else. The hybrid approach — routing each question to the right retriever, fusing the results, and structuring the context for the LLM — wins everywhere.
This guide shows you exactly how to build that hybrid system.
What You'll Build
User Question → Query Router → [Vector Retriever | Graph Traverser | Both] → Context Fuser → LLM
↓ ↓
Embedding Store Neo4j KG
Chapter 1: Architecture & Query Routing
How to classify incoming questions as vector-destined, graph-destined, or hybrid — and route to the appropriate retriever without the LLM in the loop. Includes a decision tree that handles edge cases (ambiguous questions, mixed intent, no-match).
Chapter 2: Hybrid Retrieval Engine
The core fusion logic: run vector and graph retrieval in parallel, rank results by relevance, deduplicate across sources, and serialize into structured LLM context that preserves relationship metadata.
Chapter 3: Entity Resolution Pipeline
LLM-extracted entities are dirty. "Amazon", "Amazon.com", "AMZN", "Amazon Web Services" — these should be one node. This chapter covers deduplication with fuzzy matching, alias resolution, and confidence scoring before Neo4j insertion.
Chapter 4: Context Serialization
How to serialize Neo4j traversal paths as LLM-readable context. Not just MATCH (a)-[:WORKS_AT]->(b) RETURN a,b — but structured context that tells the LLM "Alice works at Acme Corp (acquired 2023, 450 employees) which has an office in Berlin."
Chapter 5: Incremental Updates
Your graph grows. New entities appear. Relationships change. This chapter covers adding nodes and edges without rebuilding embeddings, updating vector indexes incrementally, and handling conflicting updates from multiple sources.
Chapter 6: Performance Runbook
Connection pooling configuration, index strategy selection, query profiling with PROFILE, cold-start vs warm-cache benchmarks, and the three queries that kill Neo4j performance at scale (and how to rewrite them).
Chapter 7: Evaluation Framework
How to measure whether your hybrid system actually performs better than vector-only. Precision, recall, MRR, NDCG, hallucination rate, and the "hard question set" that exposes weaknesses in your retrieval pipeline.
Chapter 8: Implementations
Five complete Python implementations:
- LangChain custom retriever class
- Raw Neo4j driver with Cypher templates
- FastAPI retrieval endpoint
- Batch evaluation script
- Incremental update daemon
What You'll Be Able To Do
After reading this guide and implementing the code:
- Answer multi-hop questions that pure vector RAG cannot — "Find all suppliers of product X who also supply to our top 3 competitors"
- Route queries intelligently — vector search for similarity, graph traversal for relationships, hybrid for complex intent
- Keep your graph clean — entity resolution pipeline catches duplicates before they corrupt your schema
- Measure actual improvement — evaluation framework proves your hybrid approach beats vector-only with numbers
- Ship incrementally — add new data without rebuilding embeddings or re-indexing
Who This Is For
- RAG engineers who've hit the ceiling on semantic similarity and need structured reasoning
- Teams using LangChain or LlamaIndex who want to add graph-backed retrievers without framework lock-in
- Neo4j developers who have graph data and want to connect it to LLM applications
- Anyone who's read the GraphRAG paper and wondered "okay but how do I actually build this"
Format & Delivery
PDF, approximately 58 pages. Includes runnable Cypher queries, Python code (type-annotated), and evaluation scripts. Immediate digital download.
Before You Buy
The query router decision tree from Chapter 1 is available as a free preview →. It shows you exactly how to classify queries and route to the right retriever. Read it, implement it, and if the hybrid approach improves your retrieval accuracy, come back for the remaining chapters.
