Multimodal Knowledge Graphs: When the Graph Learns to See and Hear

For most of their history, knowledge graphs were effectively text databases with structure. An entity was a label, a relation was a string, and the "knowledge" was what a paper could say about the world. That assumption is collapsing in 2026. The newest research embeds images, audio, video, 3D geometry and spatial context directly into graph structure — creating the multimodal knowledge graph (MMKG). It is a quiet but broad shift, bursting strongly across graph and slash-LLM research, and the corpus now contains enough construction, completion, retrieval and benchmarking work to make a clear picture.

Why multimodal matters

A text-only KG cannot represent the thing most of the world's data actually is. What an agent needs to know about a product, a drug, a film or a building lives in images, sensor streams and recorded speech. Two forces drive the move to MMKGs:

• Multimodal LLMs got good enough to ground learned relationships in images and audio, not just in text; and
• Multimodal GraphRAG needs a structured home for the mixed evidence it unifies, across languages, audio and rendering.

The result is that an MMKG is no longer a research novelty — it is the natural data fabric for agentic retrieval over the world's real, non-text data.

Building them: construction and completion

Construction moved from hand-labelling to automatic, evidence-grounded pipelines:

• Evidence-Grounded Multimodal Knowledge Graph Construction — builds MMKGs for multi-lecture educational reasoning, creating graph nodes from repeated educational media of both slides and spoken lecture content, each edge carrying the evidence it was grounded on.
• Dual-Path LLM Reasoning for Multimodal Few-Shot KG Completion — completion is the hard part (new entities, few-shot and zero-shot relations), and this paper fuses multimodal LLM-derived priors with factual support to complete sparse multimodal graphs.
• MGDT — an MLLM-guided diffusion transformer with relation-adaptive mixture-of-experts, generating or completing multimodal node representations.
• EvoGraph-R1 — self-evolving multimodal knowledge hypergraphs for agentic retrieval, showing the graph can grow relationships across modalities as the agent reasons.

Retrieval over multimodal graphs

The application layer is where it becomes useful. Multimodal GraphRAG can answer questions spanning text and render.

• MKG-RAG-Bench — the emerging benchmark for retrieval in multimodal knowledge graph-augmented generation, giving the field a common measurement.