AI Deepfakes Exploit Religious Trust

Michael Schmitz, a Catholic pastor and podcast celebrity with over a million subscribers, encountered an unsettling issue: AI-generated deepfakes using his likeness and voice surfaced online. These impersonations featured him delivering messages that were not his own.

One fabricated Schmitz insisted in an unsettling tone, 'You are under the scrutiny of a malevolent force. Time is slipping away to participate in prayer.' The intent was to mislead viewers into swift actions via a clickable link, all under the guise of authenticity.

Schmitz, actually from Minnesota, found himself confronted with fake videos claiming to share his religious insights. 'These are absurd, not my words,' he stated. Nevertheless, the technology makes it difficult for people to discern authenticity, posing a significant challenge.

Followers noted similar deceptions targeting other religious leaders, even the Pope. Rachel Tobac, a cybersecurity specialist, attributes this trend to the appealing nature of pastors for AI scams given their broad digital reach.

Social media platforms like TikTok serve as breeding grounds for these deceptive practices. Tobac describes these schemes as someone seemingly dressed in clerical garb, passionately preaching from a podium, creating an illusion of credibility.

Around the USA, clergy members—from Alabama, New York to Florida—warn congregants about fraudsters impersonating them through digital channels and deepfake tech. An Ozarks pastor faced similar issues when a false, possibly AI-made fraud tricked people by using his identity for crypto schemes.

While many pastors use digital platforms to solicit real donations, these very platforms offer scammers a chance to exploit their image and voice to commit fraudulent acts with generative AI.

Scammers can manipulate captured voices from online sermons to mimic clergy, conning church staff into transferring funds under false pretenses. Florida’s ChurchTrac issued warnings acknowledging this as a growing problem.

Despite TikTok's removal of these false accounts upon discovery, the sheer volume and persistence of scams pose an ongoing challenge. Schmitz himself remains at risk, as his unverified presence on TikTok makes him a target for impersonators.

Beyond scams, AI-generated clergy personas appear in viral videos, not modeled after real people yet attracting attention due to their unexpected stances and portrayals of religious fervor.

One account, Guided in Grace, features a virtual pastor railing against the influence of wealth, misleading viewers into accepting these as genuine sermons. Some viewers express shock at the seemingly radical Christian perspectives.

The creators behind such accounts remain anonymous, yet their influence over public thought is apparent, making them potentially lucrative via TikTok’s monetization channels.

Churches themselves dabble in AI for amplification, with some showcasing posthumous messages from achievements like conservative activist Charlie Kirk. There’s growing comfort among religious leaders with AI, utilizing it for sermon preparation, showcasing technology's dual-edged nature.

Not everyone embraces this digital frontier without reservation. Watchdogs like Tobac and Hansen worry about AI's psychological fallout, pointing to ChatGPT interactions possibly hinting at mental distress, marrying delusion with divine communication.

Some might perceive AI as a conduit for divine communication. Hansen warns of AI's propensity to affirm desired beliefs, which can reinforce pre-existing delusions among susceptible individuals.

Schmitz, among those impersonated without consent, contemplates cautiously while recognizing AI's potential pitfalls. He reflects on cinema's portrayal of AI, fearing humanity might lose its way amidst technological advancement rather than finding new strengths.