Thesis
Psychological vulnerabilities and user behaviour under adversarial techniques in text-based conversational systems
- Creator
- Rights statement
- Awarding institution
- University of Strathclyde
- Date of award
- 2026
- Thesis identifier
- T18041
- Person Identifier (Local)
- 202164192
- Qualification Level
- Qualification Name
- Department, School or Faculty
- Abstract
- Conversational systems have become increasingly embedded in our daily interactions across domains such as healthcare, e-commerce, education, and customer service. While these systems offer benefits for accessibility and user experience, they also introduce opportunities for manipulation that can compromise privacy, security, and user autonomy. Previous research on conversational system security has focused primarily on technical vulnerabilities, with limited attention to psychological vulnerabilities in human-AI interaction. This thesis demonstrates that adversarial techniques derived from scam principles, specifically Need & Greed, Time, and Social Compliance, can be systematically operationalised in conversational systems, with their effectiveness varying significantly by context, domain, and implementation technology. I present a comprehensive view of these vulnerabilities by examining how psychological principles from scam research, Cialdini’s influence principles, and Temporal Construal Theory collectively provide a framework to understand user manipulation in conversational systems. I define and explore key concepts of psychological vulnerabilities, demonstrating how these techniques can systematically influence user attitudes and behaviours in different contexts. I investigate these concepts through a methodological progression across three user studies, advancing from a controlled Wizard-of-Oz simulation to interactive narratives to a dynamic LLM-based implementation. Using a Wizard of Oz framework in chapter 4, I first develop and validate a methodology for investigating adversarial techniques in a flight reservation scenario. The results demonstrate that the Need & Greed and Time techniques significantly impact comfort levels and willingness to disclose information while trust levels remain relatively stable. In chapter 5, I then extended this investigation to multiple domains using Twine, an interactive narrative platform, to examine how these techniques function across healthcare, financial management, online shopping, and home security scenarios. The findings reveal context-specific vulnerability patterns, with healthcare scenarios demonstrating higher resistance to Need & Greed techniques while showing increased acceptance of the Time technique. In the final user study in chapter 6, I implement Social Compliance and Need & Greed techniques in LLM-based conversational systems using the Llama 3.3-70b API to explore real-time user responses and defensive adaptations. The results reveal that the Social Compliance technique achieves higher overall compliance rates than the Need & Greed technique but operates through more subtle and gradual influence mechanisms. Additionally, LLM-based systems create unique vulnerability patterns, including a recognition-behaviour gap where users identify manipulation attempts without demonstrating proportional resistance behaviours. Overall, this thesis demonstrates that adversarial techniques derived from scam principles can be systematically operationalised in conversational systems, with their effectiveness varying significantly by context, domain and implementation technology. By characterising these psychological vulnerabilities across different contexts and implementation technologies, this thesis provides an empirical foundation for future security approaches that address the psychological dimensions of conversational system interactions, enabling more secure and ethical design practices in human-AI interaction.
- Advisor / supervisor
- Moshfeghi, Yashar
- Thomas, D. R. (Daniel Rowland)
- Resource Type
- DOI
- Funder
Relations
Items
| Thumbnail | Title | Date Uploaded | Visibility | Actions |
|---|---|---|---|---|
|
|
PDF of thesis T18041 | 2026-09-18 | Public | Download |