Schema Designed for Retrieval: Closing the KG-to-RAG Quality Gap
The graph is beautiful, complete, and authoritative. And the RAG answers are worse than a plain vector index. This is the most common and most demoralizing outcome of a GraphRAG build: the knowledge graph stores everything correctly but retrieves nothing usefully. The cause is almost never effort — it's that the schema was designed for storage, not for retrieval.
The storage-first schema trap
When you model a graph the way you store data, you optimize for canonical correctness: every fact stored once, normalized, no redundancy. That is a great data model and a poor retrieval model. Retrieval is not about where a fact belongs; it's about how fast and how fully an LLM can reach the facts it needs to answer a question, in the order it needs them.
A vector store answers a question by semantic similarity to a chunk. A graph answers by traversal from a starting node. If your schema has no natural entry point for the question ("which vendor does Tobi's team depend on for compute?"), your retrieval has to start somewhere unhelpful and traverse a long, indirect path — or give up.
Model around the questions, not just the facts
Before touching the schema, collect the actual questions the system must answer, then ask what path through the graph answers each one. That transforms modeling into a design exercise with a target:
- Questions are entity-first. Almost every query starts with "give me the one entity X". Make sure X is directly addressable and indexable — a search entry point, not buried behind a traversal.
- Answer paths need to be short and typed. If "all compute dependencies
of Tobi's team" requires five hops, consider a typed shortcut edge
(
(:Team)-[:DEPENDS_ON]->(:Vendor)) even though it's redundant with a longer path. Redundancy is a feature when its purpose is retrieval speed. - Prefer typed relationships over generic
RELATED_TO. A generic edge means the LLM must infer semantics at query time. ADEPENDS_ON,AUTHORED_BY,PURCHASED_FROMedge lets you navigate by meaning and keeps generated traversal cheap and correct.
The checklist that catches the gap
When retrieval underperforms, walk this list before tuning embeddings:
- Is there a direct entry point for the top questions? If a question's subject isn't indexable/searchable directly, add one.
- Are your answer paths too deep? Compute the average hops for the top-20 questions. Long paths signpost a missing shortcut edge.
- Do you rely on generic edges? Replace the ones your queries actually traverse with typed, query-specific relationships.
- Is critical context one hop from the entity? LLMs prefer the facts bundled close to the subject — an entity node that links to an isolated fact island will not surface it.
Redundancy is okay when it buys retrieval
Storage-first instincts resist a shortcut edge that duplicates a traversal. Retain them anyway: a retrieval-tuned graph is allowed, even encouraged, to be slightly denormalized if it shortens the path from a question to its answer. The rule is simple — canonical data lives in the durable layer; the retrieval graph adds the shortcuts that make answers reachable.
Measure with end-to-end answers, not graph stats
Judge the schema by the answers, not by pedigree. Build a small golden set of the top questions, run your RAG against both the old schema and the new, and compare. If the graph still doesn't beat a vector-only baseline on answers that need joined facts, the schema hasn't been tuned for retrieval yet.
The difference between a storage-first and a retrieval-first schema is the difference between a museum and a map: one catalogues everything accurately, the other helps you actually get somewhere. Model for the questions, add typed shortcuts, keep answers one hop away — and your graph stops being a beautiful liability and starts winning the retrieval battles it was built for.