Semester of Graduation
Summer 2026
Degree Type
Dissertation/Thesis
Degree Name
(M.S.) in Information Technology
Department
Information Technology
Committee Chair/First Advisor
Dr. Amir Karami
Second Advisor
Dr. Muhammad Imran
Third Advisor
Dr. Dhrubajyoti Ghosh
Abstract
Generative artificial intelligence has unsettled a core assumption of course assessment: that scores on unsupervised work reflect what students can do unassisted. This study tests whether the widespread availability of ChatGPT altered student performance differently on formative versus summative assessments in Business Analytics and Foundations, a required undergraduate quantitative-methods course taught by one instructor across nine cohorts. In a quasi-experimental longitudinal design, the Fall~2022 cohort ($n = 90$), the last to finish before ChatGPT's public release, serves as a control against eight post-ChatGPT cohorts spanning Spring~2023 through Summer~2025 ($N = 646$). Outcomes were drawn from McGraw-Hill Connect records using matched assessment items, percentage normalization, and behavioral traces such as attempt counts and time on task. Given bounded, skewed, ceiling-tied distributions, every outcome was evaluated with the Jonckheere--Terpstra test for ordered alternatives, with each $p$-value validated against a 20{,}000-iteration label-permutation null. The results reveal a consistent formative--summative divergence. On unsupervised formative work, reading and the pooled quiz-plus-exercise score both rose significantly, while effort measures moved in the predicted direction without reaching significance. The proctored online examination, which blocks real-time AI access, declined slightly but significantly, and exam time per question drifted marginally upward. The pooled quiz-plus-exercise median rose from $93.32\%$ to $96.36\%$ while the proctored exam median fell from $77.09\%$ to $68\%$. Apparent formative gains do not transfer to unassisted summative performance, with direct implications for how institutions design, weight, and interpret assessments.
Thesis Defense Signed Form
Thesis Defense Signed.pdf (243 kB)
Thesis Defense Signed Form