X-KGRank: When Small Language Models Lie (And How Knowledge Graphs Fix It)

Modern recommender systems produce predictions that users cannot interrogate. Collaborative filtering captures behavioural signals but offers no reasoning. Large Language Models generate fluent explanations but hallucinate and are poorly grounded in user history.

X-KGRank (Explainable Knowledge Graph RAG) unifies structural collaborative filtering with LLM-based explanation. From the MovieLens-1M dataset, it constructs a heterogeneous knowledge graph of 9,762 nodes and 999,264 edges. It trains a LightGCN ranker with content-aware SBERT initialization and applies popularity selective routing that grounds long-tail items in knowledge-graph paths while serving popular items from pre-trained knowledge.

The result: NDCG@10 = 0.2956, +17.1% over popularity baseline, and a critical finding — smaller models match larger models on explanation quality but fabricate facts more often.

The Explainability Problem

Traditional recommender systems:

User → Collaborative Filtering → Recommendation
       (matrix factorization, LightGCN, etc.)
       Output: "You might like Movie X"
       Explanation: None (or "because you liked Y")

LLM-based recommenders:

User → LLM → Recommendation + Explanation
       Output: "You might like Movie X because..."
       Explanation: "Based on your love of sci-fi and strong female leads..."
       Problem: Hallucinated, not grounded in actual history

The gap: Collaborative filtering is accurate but unexplainable. LLMs are explainable but hallucinate.

X-KGRank bridges this gap by grounding explanations in knowledge graph structure.

X-KGRank Architecture

1. Heterogeneous Knowledge Graph Construction

# From MovieLens-1M (6,040 users, 3,704 items, 988,129 interactions)
class X_KGRankKG:
    def __init__(self, ratings_data):
        self.graph = Neo4jGraph()
        
        # Create user nodes
        for user_id in ratings_data.users:
            self.graph.create_node('User', user_id, metadata={
                'rating_count': len(ratings_data.get_user_ratings(user_id))
            })
        
        # Create item nodes
        for item_id in ratings_data.items:
            self.graph.create_node('Item', item_id, metadata={
                'title': ratings_data.get_item_title(item_id),
                'genres': ratings_data.get_item_genres(item_id)
            })
        
        # Create rating edges
        for user_id, item_id, rating in ratings_data.interactions:
            self.graph.create_edge('User', user_id, 'RATED', 'Item', item_id, {
                'rating': rating,
                'timestamp': ratings_data.get_timestamp(user_id, item_id)
            })
        
        # Create genre edges
        for item_id, genres in ratings_data.item_genres.items():
            for genre in genres:
                self.graph.create_edge('Item', item_id, 'HAS_GENRE', 'Genre', genre)
        
        # Create co-rating edges (popularity signal)
        item_counts = ratings_data.get_item_popularity()
        popularity_threshold = 100  # Items with >100 ratings are "popular"
        for item_id, count in item_counts.items():
            if count > popularity_threshold:
                self.graph.create_edge('Item', item_id, 'CO_RATED', 'Item', item_id, {
                    'co_rating_count': count
                })

class LLM_ReRanker: def __init__(self, llm, kg): self.llm = llm self.graph = kg # Knowledge graph for path finding def rerank(self, candidates, user_history): """Re-rank candidates with LLM scoring""" scored = [] for item in candidates: # Generate explanation explanation = self._generate_explanation(item, user_history) # Score explanation quality quality = self._score_explanation(explanation) scored.append({ 'item': item, 'explanation': explanation, 'quality': quality }) # Sort by quality scored.sort(key=lambda x: x['quality'], reverse=True) return scored def _generate_explanation(self, item, user_history): # Query KG for paths between user history and item paths = self.graph.find_paths( start=user_history.items, end=item, max_hops=3 ) # Generate explanation from paths prompt = f""" User has rated: {user_history.top_items} Recommended item: {item.title} Knowledge graph paths: {self._format_paths(paths)} Generate a natural explanation for this recommendation. """ explanation = self.llm.generate(prompt) return explanation def _format_paths(self, paths): """Format KG paths into human-readable text for the LLM prompt.""" lines = [] for path in paths: hops = " → ".join(str(node) for node in path) lines.append(f" - {hops}") return "\n".join(lines) def _score_explanation(self, explanation): # Heuristic scoring (0-1) # Based on: specificity, grounding, coherence score = 0 # Check for specific item references if len(explanation.split()) > 20: score += 0.3 # Check for user history mentions if 'because you' in explanation.lower(): score += 0.3 # Check for knowledge grounding if 'similar to' in explanation.lower() or 'also' in explanation.lower(): score += 0.4 return score

Metric	Popularity Baseline	X-KGRank	Improvement
NDCG@10	0.2525	0.2956	+17.1%
Recall@10	0.4615	0.5371	+16.4%
NDCG@20	0.2983	0.3449	+15.6%
MRR	0.2124	0.2435	+14.6%

Model	Parameter Count	Explanation Quality	Fact Fabrication Rate
Qwen2.5	1.5B	0.97	12%
Mistral	7B	0.94	8%
Llama3	8B	0.95	7%

X-KGRank: When Small Language Models Lie (And How Knowledge Graphs Fix It)

The Explainability Problem

X-KGRank Architecture

1. Heterogeneous Knowledge Graph Construction

2. LightGCN Ranker with Content-Aware Initialization

3. Popularity Selective Routing

4. LLM Re-Ranking with Explanation Generation

What the Benchmarks Say

Production Considerations

Neo4j Query Patterns

Cost Optimisation

Scaling to Millions of Users

Where the Field Is Heading

1. Hybrid Recommendation Is Standard

2. Model Size vs. Factuality Trade-off

3. Selective Routing for Cost Control

The Bottom Line

Further Reading