ifkad articles

Explainable AI as Decision Support in Automated Assessment: An Exploratory Study

Gabriele Creta, Andrea De Mauro
LUISS Guido Carli University, Rome, Italy

This paper investigates how rubric-based Explainable Artificial Intelligence (XAI) outputs influence instructor trust and grading behavior in AI-assisted academic assessment. As AI technologies become increasingly integrated into educational evaluation workflows, understanding the cognitive mechanisms through which instructors interact with AI-generated scores is critical for designing effective human–AI collaboration in grading and for the broader governance of knowledge assets in higher education (Khosravi et al., 2022). Two competing hypotheses motivate the study: explanations may either mitigate automation bias by enabling critical evaluation, or amplify it by providing a rationalization for uncritical acceptance of AI recommendations. We present a within-subject pilot experiment in which university-level evaluators (n = 3) assessed 12 short student responses to Computer Science questions under three conditions: (a) AI-only, where only a numerical AI score was displayed; (b) AI + Explanation, where the AI score was accompanied by a four-criterion rubric breakdown and a textual justification; and (c) Anchoring, where evaluators first graded independently and then were exposed to the AI output with an option to revise. AI grading outputs were generated using a Retrieval-Augmented Generation (RAG) pipeline calibrated against three open-access essay scoring datasets (GRE Analytical Writing, DREsS, and ASAP2) and indexed in a FAISS vector store for similarity retrieval. Preliminary results from this pilot are consistent with rubric-based explanations reducing deviation between human and AI grades (mean |delta| = 1.2 vs. 1.9 for AI-only), doubling the observed revision rate in the anchoring condition (67% vs. 33%), and shifting all observed revisions toward the AI score. Qualitative evidence indicates that evaluators cite specific AI observations when revising with explanations but resist adjustment without them. A discrepancy between self-reported and observed AI influence further suggests a metacognitive blind spot consistent with the automation-bias literature. These preliminary findings are consistent with the hypothesis that structured XAI outputs may enhance evaluator reliance on AI grading systems and reshape evaluative judgment through anchoring mechanisms, with implications for the governance of human–AI assessment workflows in higher education.

IN: Proceedings IFKAD 2026: Intelligent Knowledge for Sustainable Organizations
ISBN: 978-88-96687-21-5
|| ISSN: 2280-787X
PP: 529-536