Semester of Graduation
Summer 2026
Degree Type
Dissertation
Degree Name
Doctor of Philosophy in Data Science and Analytics
Department
School of Data Science and Analytics - College of Computing and Software Engineering
Committee Chair/First Advisor
Kevin Gittner
Second Advisor
Lauren Matheny
Third Advisor
Nasrin Dehbozorgi
Abstract
Large language models (LLMs) are increasingly proposed as substitutes for human survey respondents, promising scalable, low-cost data. Prior work has demonstrated individual capabilities in isolation or treated completion pipelines as black boxes, leaving no integrated account of when synthetic respondents produce valid data. This dissertation evaluates that proposition across three validity components, adapting its designs as model capabilities evolved: whether LLM agents can complete real surveys autonomously, whether their responses can be made internally coherent, and whether the personas that assign agents demographic identities reflect real populations. The first study built a modular agent that completed Qualtrics surveys on consumer-grade hardware with a locally deployed Llama 3.1-8B model, answering every question across six formats. It also identified an inadvertent assistance dynamic, in which platform validation logic intended to help human respondents, instead guided the agent toward compliant submissions. The second study addressed the statelessness of LLM inference, comparing four memory architectures across surveys of 10 to 100 questions. Full-state JSON injection and summarization strategies reached 98% internal consistency, with summarization at one-third of JSON’s token cost. The third study audited 10,138 synthetic personas from ten LLMs against U.S. Census benchmarks. Generated populations diverged substantially on age, education, and income, and most models recycled a narrow pool of popular names, though incremental generation led some models to a broader range approaching population-level variation. Together, the studies move beyond feasibility demonstrations, providing a component-level framework that specifies the technical requirements for valid synthetic respondents and establishing demographic auditing as standard practice.