K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

Analysis

Why This Paper Matters

This paper addresses a critical gap in the application of large language models to K-12 education. While existing benchmarks focus on exam question answering, they fail to assess whether LLMs understand the structured nature of curriculum knowledge—what the authors term "curriculum cognition." This includes prerequisite chains, concept taxonomies, experiment-concept links, pedagogical sequencing, and visual grounding. By introducing K12-KGraph, a curriculum-aligned knowledge graph extracted from official Chinese textbooks, the authors provide a resource that can systematically evaluate and improve LLMs' ability to reason about educational content in a structured way.

The significance is twofold. First, the benchmark K12-Bench reveals that even state-of-the-art models like Gemini-3-Flash and Gemma-4-31B-IT struggle with tasks requiring understanding of prerequisite relationships and neighbor concepts, achieving only 57% and 46% exact match respectively. This highlights a fundamental weakness in current LLMs' educational reasoning. Second, the training corpus K12-Train demonstrates that domain-specific supervision can substantially close this gap, outperforming general instruction-tuning datasets on downstream educational benchmarks. This suggests that curriculum-aligned data is more effective for educational tasks than generic data, even when sample sizes are matched.

Technical Contributions

The paper makes several key technical innovations:

• Knowledge Graph Construction: K12-KGraph is built from official People's Education Press textbooks across four subjects (mathematics, physics, chemistry, biology) and three school levels (primary, middle, high). It defines nine node types (e.g., concept, experiment, figure) and fourteen relation types (e.g., prerequisite, taxonomy, visual-grounding), capturing both curriculum structure and visual elements.
• Benchmark Design: K12-Bench comprises 23,640 multi-select questions across five task families: Ground (visual grounding), Prereq (prerequisite chains), Neighbor (concept neighbors), Evidence (experiment-concept links), and Locate (pedagogical sequencing). This allows fine-grained evaluation of different aspects of curriculum cognition.
• Training Corpus: K12-Train includes 7,335 samples, with 2,267 text-only QA pairs and 5,068 multimodal VQA pairs, all derived from the graph. The graph-guided construction ensures that training data covers diverse curriculum relationships.
• Evaluation Protocol: The authors compare K12-Train against eight mainstream instruction-tuning corpora (e.g., DataFlow, WizardLM) under matched sample budgets, and evaluate on multiple educational benchmarks (GaokaoBench, EduEval, Gaokao-MM, MDK12-medium, K12Vista).

Results

Key quantitative results include:

• On K12-Bench, Gemini-3-Flash achieves 57% exact match, Gemma-4-31B-IT reaches 46%, with Prereq and Neighbor tasks being the most challenging.
• Under a matched 2,300-sample budget, K12-Train-Text consistently outperforms equally sized subsets of eight mainstream instruction-tuning corpora on GaokaoBench and EduEval.
• For vision-language models, K12-Train-Full achieves the best overall results on Gaokao-MM, MDK12-medium, and K12Vista among all compared training configurations, despite using fewer samples than full DataFlow and WizardLM baselines.
• K12-Train-Full surpasses both text-only and multimodal-only variants, demonstrating that textual and visual supervision are complementary.

Significance

This work has broad implications for AI in education. By providing a structured knowledge graph and associated benchmarks, it enables researchers to systematically evaluate and improve LLMs' understanding of curriculum structure—a capability essential for intelligent tutoring systems, automated lesson planning, and personalized learning. The finding that domain-specific supervision outperforms general instruction-tuning data suggests that future educational AI systems should be trained on curriculum-aligned data rather than generic corpora. The public release of all resources (graph, benchmark, training data, pipeline) lowers the barrier for further research and development in this area. However, the focus on Chinese textbooks limits immediate applicability to other educational systems, and future work should explore cross-curriculum generalization.