Journal Article
Large Language Models

Unifying Large Language Models and Knowledge Graphs: A Roadmap

Shirui Pan(Griffith University), Linhao Luo(Monash University), Yufei Wang(Monash University), Chen Chen(Nanyang Technological University), Jiapu Wang(Beijing University of Technology), Xindong Wu(Hefei University of Technology)
January 10, 2024IEEE Transactions on Knowledge and Data Engineering1,877 citations
arXivPDFDOI

1.9k

Citations

74

Influential Citations

IEEE Transactions on Knowledge and Data Engineering

Venue

2024

Year

Abstract

Large language models (LLMs), such as ChatGPT and GPT4, are making new waves in the field of natural language processing and artificial intelligence, due to their emergent ability and generalizability. However, LLMs are black-box models, which often fall short of capturing and accessing factual knowledge. In contrast, Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge. KGs can enhance LLMs by providing external knowledge for inference and interpretability. Meanwhile, KGs are difficult to construct and evolve by nature, which challenges the existing methods in KGs to generate new facts and represent unseen knowledge. Therefore, it is complementary to unify LLMs and KGs together and simultaneously leverage their advantages. In this article, we present a forward-looking roadmap for the unification of LLMs and KGs. Our roadmap consists of three general frameworks, namely,1) KG-enhanced LLMs,which incorporate KGs during the pre-training and inference phases of LLMs, or for the purpose of enhancing understanding of the knowledge learned by LLMs;2) LLM-augmented KGs,that leverage LLMs for different KG tasks such as embedding, completion, construction, graph-to-text generation, and question answering; and3) Synergized LLMs + KGs, in which LLMs and KGs play equal roles and work in a mutually beneficial way to enhance both LLMs and KGs for bidirectional reasoning driven by both data and knowledge. We review and summarize existing efforts within these three frameworks in our roadmap and pinpoint their future research directions.

Analysis

Why This Paper Matters

Large language models (LLMs) like ChatGPT and GPT-4 have revolutionized natural language processing with their emergent abilities and generalizability. However, they are black-box models that often struggle to capture and access factual knowledge, leading to issues such as hallucination and lack of interpretability. Knowledge graphs (KGs), on the other hand, explicitly store rich factual knowledge in a structured form, providing a complementary strength. This paper argues that unifying LLMs and KGs is mutually beneficial: KGs can enhance LLMs by providing external knowledge for inference and interpretability, while LLMs can aid KGs in tasks like construction, completion, and evolution. Given the growing interest in neuro-symbolic AI and knowledge-enhanced LLMs, this roadmap arrives at a critical juncture, offering a structured framework to guide future research.

The paper's significance is further underscored by its comprehensive review of existing efforts across three frameworks, which helps researchers navigate the rapidly expanding literature. By identifying future research directions, it sets an agenda for the community to build more robust, factual, and interpretable AI systems. With 1877 citations, it has already become a key reference in the field, reflecting its impact on both academia and industry.

Technical Contributions

  • Three unified frameworks: The roadmap categorizes the integration of LLMs and KGs into (1) KG-enhanced LLMs, where KGs are incorporated during pre-training, inference, or for enhancing understanding; (2) LLM-augmented KGs, where LLMs are leveraged for KG embedding, completion, construction, graph-to-text generation, and question answering; and (3) Synergized LLMs + KGs, where both play equal roles in bidirectional reasoning driven by data and knowledge.
  • Systematic review: The paper surveys existing methods within each framework, providing a structured overview of the state of the art.
  • Future directions: It pinpoints research gaps and proposes concrete directions for each framework, such as improving knowledge injection techniques, developing robust evaluation metrics, and enabling bidirectional reasoning.
  • Complementarity analysis: It highlights the strengths and weaknesses of LLMs and KGs, justifying the need for unification.

Results

As a roadmap and survey paper, it does not present new experimental results. Instead, it synthesizes findings from existing literature, categorizing methods and identifying trends. The paper's "results" are the conceptual frameworks and the research agenda they enable. It does not provide quantitative comparisons or benchmarks, which is typical for position papers. However, the frameworks have been widely adopted by subsequent work, as evidenced by the high citation count.

Significance

The paper's significance lies in its role as a foundational reference for unifying LLMs and KGs. It provides a clear taxonomy that helps researchers position their work and identify open problems. By advocating for synergized LLMs+KGs, it promotes the development of AI systems that combine the flexibility of neural models with the precision of symbolic knowledge, potentially leading to more reliable and interpretable AI. The roadmap also has implications for industry applications, such as question answering, recommendation systems, and drug discovery, where factual accuracy and interpretability are crucial. Overall, this paper is a timely and influential contribution that shapes the direction of knowledge-enhanced LLM research.

large-language-modelsknowledge-graphsneuro-symbolicsurveyroadmapnatural-language-processingknowledge-representation
Large Language ModelsReasoning

Related Papers