Faced with mounting backlash, OpenAI removed a controversial ChatGPT feature that caused some users to unintentionally allow their private—and highly personal—chats to appear in search results.
Fast Company exposed the privacy issue on Wednesday, reporting that thousands of ChatGPT conversations were found in Google search results and likely only represented a sample of chats "visible to millions." While the indexing did not include identifying information about the ChatGPT users, some of their chats did share personal details—like highly specific descriptions of interpersonal relationships with friends and family members—perhaps making it possible to identify them, Fast Company found.
OpenAI's chief information security officer, Dane Stuckey, explained on X that all users whose chats were exposed opted in to indexing their chats by clicking a box after choosing to share a chat.
Fast Company noted that users often share chats on WhatsApp or select the option to save a link to visit the chat later. But as Fast Company explained, users may have been misled into sharing chats due to how the text was formatted:
"When users clicked 'Share,' they were presented with an option to tick a box labeled 'Make this chat discoverable.' Beneath that, in smaller, lighter text, was a caveat explaining that the chat could then appear in search engine results."
At first, OpenAI defended the labeling as "sufficiently clear," Fast Company reported Thursday. But Stuckey confirmed that "ultimately," the AI company decided that the feature "introduced too many opportunities for folks to accidentally share things they didn't intend to." According to Fast Company, that included chats about their drug use, sex lives, mental health, and traumatic experiences.
Carissa Veliz, an AI ethicist at the University of Oxford, told Fast Company she was "shocked" that Google was logging "these extremely sensitive conversations."
OpenAI promises to remove Google search results
Stuckey called the feature a "short-lived experiment" that OpenAI launched "to help people discover useful conversations." He confirmed that the decision to remove the feature also included an effort to "remove indexed content from the relevant search engine" through Friday morning.
Google did not respond to Fast Company's reporting, which left it unclear what role it played in how chats were displayed in search results. But a spokesperson told Ars that OpenAI was fully responsible for the indexing, clarifying that "neither Google nor any other search engine controls what pages are made public on the web. Publishers of these pages have full control over whether they are indexed by search engines."
OpenAI is seemingly also solely responsible for removing the chats, perhaps most quickly by using a tool that Google provides to block pages from appearing in search results. But that tool does not stop pages from being indexed by other search engines, so it's possible chats will disappear sooner in Google results than other search engines.
Véliz told Fast Company that even a "short-lived" experiment like this is "troubling," noting that "tech companies use the general population as guinea pigs," attracting swarms of users with new AI products and waiting to see what consequences they may face for invasive design choices.
"They do something, they try it out on the population, and see if somebody complains," Véliz said.
To check if private chats are still being indexed, a Fast Company explanation suggests that users who still have access to their shared links can try inputting the "part of the link created when someone proactively clicks 'Share' on ChatGPT [to] uncover conversations" that may still be discoverable on Google.
OpenAI declined Ars' request to comment, but Stuckey's statement suggested that the company knows it has to earn back trust after the misstep.
"Security and privacy are paramount for us, and we'll keep working to maximally reflect that in our products and features," Stuckey said.
The scandal notably comes after OpenAI vowed to fight a court order that requires it to preserve all deleted chats "indefinitely," which worries ChatGPT users who previously felt assured their temporary and deleted chats were not being saved. OpenAI has so far lost that fight, and those chats will likely be searchable soon in that lawsuit. But while OpenAI CEO Sam Altman considered the possibility that users' most private chats could be searched to be "screwed up," Fast Company noted that Altman did not seem to be as transparently critical about the potential for OpenAI's own practices to expose private user chats on Google and other search engines.
By Ashley Belanger - Senior Policy Reporter
