Abstract

Explainable Artificial Intelligence (XAI) is rapidly being integrated into project and program management decision-support tools, driven by the widely held assumption that model transparency improves human decision quality. This assumption has not been subjected to rigorous empirical testing in program management contexts. This paper presents the design, theoretical framework, experimental methodology, and pilot study findings of a controlled behavioural experiment evaluating whether program managers who receive XAI-augmented risk predictions make measurably better decisions than those receiving prediction-only outputs or no AI assistance. Three experimental conditions are defined: unaided judgment, prediction-only AI output, and XAI-augmented AI output with SHAP-derived feature explanations. Four dependent variables are measured: decision accuracy, intervention targeting precision, decision latency, and stakeholder communication quality. A pilot study conducted with 18 practitioners provides preliminary support for the primary hypotheses, with moderate-to-large effect size estimates confirming adequate statistical power for the full study. The framework is grounded in Dual Process Theory, Bounded Rationality, and the Technology Acceptance Model. The study addresses a critical empirical void at the intersection of explainable AI and program governance and is positioned to produce the first empirically validated evidence base on the decision-level value of XAI in IT program risk management.

This work is licensed under a Creative Commons Attribution 4.0 License.