Graph RAG for automated short answer grading with feedback: bridging pedagogical needs and technical capabilities

Analysis

Why This Paper Matters

Automated short answer grading (ASAG) is a critical yet challenging task in education technology. Traditional methods often rely on lexical matching or opaque neural models, which fail to provide meaningful feedback to students. This paper addresses this gap by introducing a Graph RAG approach that not only grades answers but also offers explainable, fact-level feedback. The key insight is that the granularity of evidence matters: Vector-RAG retrieves unstructured passages that are hard to trace to specific claims, whereas Graph-RAG retrieves a subgraph of explicitly linked facts, making the reasoning process transparent.

This work is significant because it directly tackles the pedagogical need for actionable feedback. In real classrooms, students need to know not just whether their answer is correct, but which specific facts they missed or misstated. By leveraging knowledge graphs, the proposed method aligns the technical capability of RAG with educational requirements, potentially improving learning outcomes.

Technical Contributions

• Graph RAG Framework for ASAG: Introduces a novel application of Graph RAG to short answer grading, where evidence is retrieved as a subgraph of linked facts rather than raw text passages.
• Evidentiary Granularity Analysis: Provides a clear comparison between Vector-RAG and Graph-RAG, highlighting the trade-offs in traceability and precision.
• Bridging Pedagogy and Technology: Explicitly connects educational needs (fine-grained feedback) with technical design choices, offering a principled approach.
• Explainable Feedback Generation: The graph-based retrieval enables the generation of feedback that points to specific facts, enhancing interpretability.

Results

The abstract does not include quantitative metrics, but the qualitative comparison indicates that Graph-RAG outperforms Vector-RAG in terms of evidentiary granularity and traceability. This suggests that for tasks requiring precise, fact-level feedback, graph-based retrieval is superior. However, without concrete numbers, it is difficult to assess the magnitude of improvement or the trade-offs in retrieval accuracy.

Significance

This paper contributes to the growing intersection of knowledge graphs and large language models. By demonstrating a practical application where graph structure enhances explainability, it encourages further research into hybrid retrieval methods. For the AI community, it underscores the importance of aligning model outputs with human interpretability, especially in high-stakes domains like education. The approach could extend to other areas requiring verifiable evidence, such as legal or medical question answering.