RP35 - Impact of Anthropomorphised Assistance Systems
A critical goal for all technologies developed in WisPerMed is to instill calibrated trust in the users [1]. This means users should not trust technology too much when it is not sufficiently reliable, but they should also not distrust technology that aids their decisions. The technology must be designed to lead to appropriate reliance (see RP11, RP22). Social, anthropomorphised cues have long been shown to influence how people interact with technology, potentially increasing trust or reliance [2]. Even though AI technology anyway includes social cues by taking roles of fellow humans (e.g., by recommending treatments such as a peer doctor) and displaying increased agency [3], it can be assumed that anthropomorphised social cues such as a human-like appearance or natural language might further increase trust [4] - potentially jeopardising calibrated trust and appropriate reliance. The balance between enhancing trust and avoiding overtrust or undertrust is delicate and context-dependent. Effective design of anthropomorphised interfaces requires a deep understanding of the target user group and the specific application domain. Specifically, different user groups need to be considered, as earlier research demonstrated that various user attributes (propensity to trust, affinity for technology, medical expertise) critically influence overtrust and undertrust [5] (RP11). Therefore, a series of experimental studies will be conducted to better understand the effects of anthropomorphised social cues on various user groups. Besides users’ diversity, different forms and degrees of anthropomorphisation, the system’s transparency and ability to explain itself will be varied systematically. Besides scrutinizing the effects of the system on user experience and user reliance, potentially also physiological and neuroscientific reactions (fMRI) can be assessed. This will pave the way to providing human-like interfaces which can make decision support systems more engaging and trustworthy, but at the same time are carefully designed (in collaboration with RP36 and RP38) to ensure that users maintain a balanced and informed trust, leveraging the benefits of anthropomorphisation without falling into the pitfalls of over-reliance or excessive scepticism.
[1] M. Wischnewski, N. Krämer, and E. Müller, “Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, in CHI ’23. New York, NY, USA: Association for Computing Machinery, Apr. 2023, pp. 1–16. doi: 10.1145/3544548.3581197.
[2] C. Nass and Y. Moon, “Machines and Mindlessness: Social Responses to Computers,” J. Soc. Issues, vol. 56, no. 1, pp. 81–103, 2000, doi: 10.1111/0022-4537.00153.
[3] S. S. Sundar, “Rise of Machine Agency: A Framework for Studying the Psychology of Human–AI Interaction (HAII),” J. Comput.-Mediat. Commun., vol. 25, no. 1, pp. 74–88, Mar. 2020, doi: 10.1093/jcmc/zmz026.
[4] N. Krämer, M. Wischnewski, and E. Müller, “Interacting with autonomous systems and intelligent algorithms – new theoretical considerations on the relation of understanding and trust,” May 07, 2023, OSF. doi: 10.31234/osf.io/h32ze.
[5] A. Küper and N. Krämer, “Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI,” Int. J. Human–Computer Interact., vol. 0, no. 0, pp. 1–17, doi: 10.1080/10447318.2024.2348216.