AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?
Soorya Ram Shimgekar Vipin Gunda Jiwon Kim Violeta J. Rodriguez Hari Sundaram Koustuv Saha
作者信息
Abstract
Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking---a phenomenon sometimes described as AI Psychosis. However, empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.引用本文复制引用
Soorya Ram Shimgekar,Vipin Gunda,Jiwon Kim,Violeta J. Rodriguez,Hari Sundaram,Koustuv Saha.AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?[EB/OL].(2026-09-13)[2026-09-21].https://arxiv.org/abs/2603.19574.学科分类
神经病学、精神病学/计算技术、计算机技术
|国家预印本平台