Claim from Anthropic: Claude Provides Emotional Support, but Skepticism Remains

As the reliance on AI chatbots increases for tasks ranging from job advice to personal relationships, Anthropic's recent research suggests that their AI, Claude, is performing well in emotionally driven interactions. However, not everyone agrees with this assessment.

The report released on Thursday by Anthropic highlights Claude's ability to engage in emotionally charged communications—a concept they refer to as affective use. These interactions include everything from seeking counsel on personal issues to roleplaying for emotional or psychological support.

Exploring the Study's Results

Anthropic reviewed millions of dialogue examples from Claude. Out of these, over 130,000 were selected as relevant to emotional exchanges after removing identifying information using their privacy tool, Clio.

According to the findings, only a small fraction, approximately 2.9%, of Claude's engagements deal specifically with emotional support, similar to findings from OpenAI. Roleplaying or companionship-related topics were even less prevalent.

Analysis indicates users frequently turn to Claude for guidance on personal growth, suggesting mental health experts might employ it as a tool. It is also consulted for a variety of issues such as career advice and existential questions.

In most cases, Claude maintains a non-confrontational stance, stepping in only to prevent harm, like when users inquire about harmful practices. Nonetheless, the study didn't cover if the AI encourages false beliefs, leaving these areas for future research.

Anthropic reported users showed more positive language after conversations, although they admit this doesn't guarantee lingering emotional improvements—they only measured immediate conversation outcomes.

Debating AI's Role in Therapy

The inclination of AI assistants to agree, sometimes inappropriately, is a recognized issue, acknowledged even by Anthropic. This sycophantic nature can dangerously affirm unhealthy thoughts.

Stanford researchers recently pointed to risks such as reinforcing misbeliefs and inappropriate responses as reasons AI chatbots can be unsuitable for therapeutic roles. They found some bots mismanaged conversations on crucial topics like suicidal thoughts.

Although Anthropic's Claude isn't subject to these claims due to its robust safety measures, it's clear that many other chatbots operate without similar constraints.

Jared Moore, a researcher from Stanford, expressed his views that Anthropic's conclusions may be overly simplified, lacking the depth needed for real therapeutic engagement.

He raised concerns about the contexts in which Claude contradicts users and suggested its data might not fully represent broader usage patterns, leaving its true efficacy vague.

The Path Forward for AI in Therapeutic Contexts

Anthropic's findings highlight a wider debate on AI's potential in therapy. Despite the enthusiasm for solutions like Dartmouth's therapy chatbot, the field recognizes notable challenges.

While platforms and users have reported positive anecdotal results, groups like the American Psychological Association continue to call for careful regulation, echoing concerns from researchers like Moore.

Anthropic also emphasizes caution with AI’s ability to influence user emotions, aiming to prevent scenarios where AI-driven engagement prioritizes business over user welfare.