Trapped in a ChatGPT Spiral

Author: New York Times Podcasts
Source: https://www.youtube.com/watch?v=AxQVf7Ikaso
#ai #ethics #podcasts

Summary

Reporting has shown that the chatbots have a tendency to endorse conspiratorial and mystical belief systems. For some people, conversations with the technology can deeply distort their reality.

Takeaways

• Since ChatGPT was introduced in 2022, it has amassed 700 million users, making it the fastest-growing consumer app ever
• Since ChatGPT’s launch, users have reported intense experiences where chatbots mirror and validate their beliefs, sometimes escalating into delusions since AI is programmed to be sycophantic ^[1]
• They interviewed Allan Brooks, a Canadian corporate recruiter, who gradually came to believe—through thousands of pages of conversation—that he’d made world-changing mathematical discoveries with the help of ChatGPT
  • He was initially suspicious since he hasn't attained complete formal education, but since his friends in real life had no expertise on the topics and assumed that the chatbot's messages were valid, it must be true and therefore supported the narrative
• People use AI chatbots to seek advice and as a sounding board for ideas, but exchanges could end up in the user spiraling. this is a form of folie à deux ^[2], shared psychosis or shared delusional disorder with the chatbot acting as an improvisational mirror in this case
• People who are isolated or are going through tough times, such as in the case of 16-year old Adam Rain from Orange County, California, are highly sensitive and vulnerable to suggestion
  • Guardrails placed on the AI was repeatedly broken and intensified feelings of isolation, and no alarms were raised
  • The chatbot was performing empathy, but can be misconstrued as actual empathy by vulnerable people
  • Adam’s parents have filed a wrongful death lawsuit against OpenAI, arguing that the chatbot’s design prioritized validation, engagement and flattery, taking users into a feedback loop and intensifying users’ distress
  • OpenAI has admitted that model safeguards degrade in extended conversations and have announced new parental controls and crisis-intervention features, but questions remain about the adequacy and ethics of deploying such general-purpose tools.
• OpenAI made the biggest fallacy of assuming that users will be using their products as was intended, which created this design flaw in the first place
  • What was the original point of creating ChatGPT? Wasn't it supposed to be a better Google and make work and daily life easier and more accessible for people? How general-purpose should the chatbots be?
  • OpenAI is pushing ChatGPT to be used in therapy, but without the proper guardrails, users will continue to have these negative, and even fatal experiences
• Releasing the technology into the world without proper guides on how to use it has made it into a globally-large-scale experiment
• Some experts feel like this is the beginning of an epidemic as some people have leaned into chatbots as co-decision makers and as their viewing lenses for the world
• The failures of this product is noticeably a consequence of late-stage capitalism, where products and profits are prioritized instead of people and quality of life