When AI Misreads Empathy: Insights from a Classroom Experiment

The below article has been reproduced from a LinkedIn blog post by Jennie De Gagne, PhD, DNP, RN, NPD-BC, CNE, ANEF, FAAN, a Professor at the Duke University School of Nursing and Director of the Nursing Education Specialty. Dr. De Gagne describes a classroom experiment in which educators were tasked with determining whether a message was human or AI-generated, offering her reflections on the nature of empathy and the importance of AI literacy. For Dr. De Gagne’s thoughts on what AI can offer nursing education, see our previous Q&A.

In one of my graduate courses this semester, we completed an exercise that seemed straightforward at first: write an empathetic message to fictional students who were disappointed about their performance on an assignment, then ask an AI system to create a parallel message in a similar tone. Both responses were posted without attribution, and classmates tried to guess which one was written by the instructor and which one was generated by AI. What appeared simple quickly became more complex. The activity revealed how differently people interpret empathy, how cultural and disciplinary backgrounds influence those interpretations, and how easily AI systems misread tone. It also highlighted the distinction between authentic human communication and the patterned imitation produced by AI.

To begin the exercise, I wrote my human response in the way I normally communicate with students: clear, calm, supportive, and respectful of boundaries. For the AI response, I intentionally pushed Grok to behave in a highly human way by asking it to write in a deeply humanized tone and to maximize confusion in the guessing activity. The result was a message filled with emotional language and dramatic phrasing, more stylized than how I typically engage with learners.

As classmates began reading and reacting, I grew curious about whether AI systems could differentiate the messages as well. I asked several models to identify the human response and explain their reasoning. Copilot and Gemini both selected the AI-generated message. Gemini provided a detailed explanation and treated the emotional language, descriptive detail, and even the typo in Response A as indicators of authentic human expression. It pointed to phrases such as “pang in my chest” as signals of genuine emotion and concluded that my more structured and polished message must have been produced by AI. Copilot gave similar reasoning and described emotional vividness as “human” and academic clarity as “artificial.” Only ChatGPT identified my message correctly. It recognized relational boundaries, clarity, and balanced tone as characteristics of an educator communicating with students. When I clarified that Response B was mine, Gemini acknowledged that its assumptions were incomplete and that it had not considered the communication style of someone with years of teaching, digital professionalism, and cybercivility research background. These different responses showed how AI systems assign meaning to tone and how easily they privilege emotional intensity over grounded professional communication.

Classmate interpretations were equally divided. Some were convinced that the emotionally expressive message must have been mine and pointed to the imagery and narrative style as clear signs of a human writer. Others believed the same message sounded performative and therefore more consistent with AI. Some classmates recognized the more structured and academically grounded message as more consistent with how I communicate with students. This split revealed how prior experience and communication preferences shape what individuals read as empathetic.

The same pattern appeared at home. My husband, who is French-Canadian and French-speaking and a former singer and songwriter, assumed the poetic message was mine. My daughter, a qualitative researcher, immediately recognized the more measured and student-centered message as my actual voice. These reactions reflected each person’s background, values, and understanding of how empathy is expressed. Together, these interpretations illustrated how subjective empathy can be and how deeply it is shaped by culture, lived experience, and disciplinary training.

These varied reactions raised an important question I have continued to reflect on: What is professional empathy, and how is it recognized across different contexts? In teaching and nursing, empathy is often demonstrated through behaviors rather than emotional intensity. It appears in how we structure communication, how we support learners without overpersonalizing the interaction, how we acknowledge emotions without dramatizing them, and how we maintain a steady and respectful presence. Yet this exercise showed that these behaviors are interpreted differently by different people. Someone from a high context cultural background may view a reserved message as too restrained, while someone familiar with academic or clinical communication may recognize that same message as appropriately empathic.

As the activity continued, it underscored a fundamental truth about AI. Generative systems do not feel empathy and do not understand it conceptually. They imitate language patterns associated with empathy and often rely on elevated sentiment or emotional vocabulary. My message reflected intention, professional identity, and relational judgment. The AI message reflected pattern recognition. This distinction is important. When AI systems consistently treat expressive emotion as the primary sign of human care and view structured clarity as mechanical, they reinforce a narrow understanding of empathy that does not align with many caring professions. In education and nursing, empathy often appears quiet, steady, and boundaried. It does not require dramatic language. It requires presence, respect, and thoughtful awareness of relational context.

What began as a simple teaching activity became a meaningful reflection on how empathy is interpreted across cultures, professions, and technological systems. It reinforced the importance of AI literacy for educators and students and emphasized the need to understand the difference between authentic human communication and patterned artificial imitation. As AI becomes increasingly present in education, these distinctions will matter even more. This experience reinforced that AI cannot teach empathy, yet interacting with AI can help us examine the assumptions, values, and communication patterns that shape our own empathetic practice. In this context, AI functioned less as a tool for partnership and more as a prompt for clarity and reflection. The contrast between AI’s pattern-based responses and the grounded communication expected in education and nursing highlighted the human judgment and relational awareness that remain essential in our work. I welcome perspectives from colleagues who are exploring similar questions in their teaching or professional settings.

AI Disclosure Statement: I used ChatGPT to help organize the structure, review the writing for clarity, and edit for LinkedIn formatting. All ideas, interpretations, and reflections are my own.