Evaluating Large Language Models in Academic Assessment Design in Nuclear Engineering Education: Precision, Prompt Engineering, and Pedagogical Implications
Abstract (300 words maximum)
The integration of Large Language Models (LLMs) into academic course development represents a transformative opportunity for higher education, particularly in technical disciplines such as nuclear engineering. These models possess the capability to generate a wide range of instructional materials, including lecture content, summaries, and assessment questions, but their accuracy and pedagogical value require systematic evaluation. This study explores the application of LLMs in generating student assessment questions for two upper-level nuclear engineering courses at Kennesaw State University: Nuclear Reactor Simulation and Nuclear Fuel Cycle. The objective is to assess both the reliability and the educational quality of AI-generated questions in alignment with course learning outcomes. A structured, iterative prompt-engineering approach was implemented to refine the LLM outputs, focusing on technical correctness, conceptual depth, and clarity of phrasing. Preliminary findings indicate that once effective prompts are established, LLMs can consistently produce assessment questions that meet course objectives and maintain technical relevance. However, achieving such precision requires deliberate prompt design, iterative testing, and subject-matter oversight. Occasional factual or contextual inaccuracies were observed, emphasizing the need for human validation and expert review before classroom implementation. This research highlights that the central challenge in leveraging LLMs for academic content generation lies not in the models’ computational capacity but in developing effective prompt-engineering strategies and robust validation frameworks. By integrating these tools thoughtfully, educators can accelerate course material development while preserving technical accuracy and pedagogical integrity, offering a scalable method for supporting the evolving demands of engineering education. A portion of this abstract was aided by AI, which is aligned with the nature of the research.
Use of AI Disclaimer
yes
Academic department under which the project should be listed
SPCEET – Mechanical Engineering
Primary Investigator (PI) Name
Dr. Eduardo B. Farfan
Evaluating Large Language Models in Academic Assessment Design in Nuclear Engineering Education: Precision, Prompt Engineering, and Pedagogical Implications
The integration of Large Language Models (LLMs) into academic course development represents a transformative opportunity for higher education, particularly in technical disciplines such as nuclear engineering. These models possess the capability to generate a wide range of instructional materials, including lecture content, summaries, and assessment questions, but their accuracy and pedagogical value require systematic evaluation. This study explores the application of LLMs in generating student assessment questions for two upper-level nuclear engineering courses at Kennesaw State University: Nuclear Reactor Simulation and Nuclear Fuel Cycle. The objective is to assess both the reliability and the educational quality of AI-generated questions in alignment with course learning outcomes. A structured, iterative prompt-engineering approach was implemented to refine the LLM outputs, focusing on technical correctness, conceptual depth, and clarity of phrasing. Preliminary findings indicate that once effective prompts are established, LLMs can consistently produce assessment questions that meet course objectives and maintain technical relevance. However, achieving such precision requires deliberate prompt design, iterative testing, and subject-matter oversight. Occasional factual or contextual inaccuracies were observed, emphasizing the need for human validation and expert review before classroom implementation. This research highlights that the central challenge in leveraging LLMs for academic content generation lies not in the models’ computational capacity but in developing effective prompt-engineering strategies and robust validation frameworks. By integrating these tools thoughtfully, educators can accelerate course material development while preserving technical accuracy and pedagogical integrity, offering a scalable method for supporting the evolving demands of engineering education. A portion of this abstract was aided by AI, which is aligned with the nature of the research.