GRP-20194 Can Mental Health Apps Really Help Caregivers? Usability Findings from Human-in-the-Loop NLP and Sentiment-Aware Analytics​

GRP-20194 Can Mental Health Apps Really Help Caregivers? Usability Findings from Human-in-the-Loop NLP and Sentiment-Aware Analytics​

https://www.kennesaw.edu/ccse/events/computing-showcase/fa25-cday-program.php

Caregivers face distinctive emotional and logistical burdens, yet many mental-health apps overlook their needs and show usability issues. We introduce an automated pipeline that analyzes 317K app-store reviews from 9 apps, mapping them to Nielsen’s usability components and heuristics, together with sentiment. To assess reliability, we run a human–AI agreement study (N=50) where a domain expert (A2) and a non- expert (A1) label reviews. For heuristics, the pipeline achieves 66% exact agreement and moderate κ=0.579 with the expert, outperforming human–human agreement; components remain harder, revealing a need to refine the codebook (e.g., learnability vs satisfaction). Complementary clustering and sentiment analyses highlight recurrent issues such as interface overload and inaccessible onboarding, positioning our pipeline as a pre-annotation aid for evidence-driven app evaluation.