OpenAI released a report today detailing numerous examples of how the company has disrupted malicious uses of its AI models, including banning some accounts for how they’re using ChatGPT. In two of the case studies, the company found evidence of threat actors that seem to originate from China using or trying to use models built by OpenAI and another U.S. AI lab “…in connection with an apparent surveillance operation and to generate anti-American, Spanish-language articles.” These examples are called Peer Review and Sponsored Discontent.
Peer Review
OpenAI recently banned a set of ChatGPT accounts whose operators used the company’s models to generate descriptions similar to sales pitches of “…a social media listening tool that they claimed to have used to feed real-time reports about protests in the West to the Chinese security services.” The researchers said the threat actors also used OpenAI’s models to debug code with the apparent intention to implement such a tool, seemingly powered by a non-OpenAI model.
Ben Nimmo, a principal investigator for OpenAI, said this was the first time the company had uncovered an A.I.-powered surveillance tool of this kind, as reported by The New York Times.
OpenAI’s report stated a version of Meta’s open source Llama was one of the AI tools used to develop the operator’s code. According to Bloomberg, Meta did not confirm that its service was involved but said that if it was, it was just one of many such tools available to the users “including AI models made in China.”
Sponsored Discontent
OpenAI also banned a range of ChatGPT accounts that used its models to generate short comments in English criticizing Chinese dissident Cai Xia and long-form news articles in Spanish focusing on U.S. social and political divisions, particularly in the lead-up to the 2024 APEC Forum in Peru.
“This is the first time we’ve observed a likely Chinese influence actor successfully publishing articles in mainstream outlets in Latin America,” the OpenAI researchers said.
OpenAI’s ongoing efforts to disrupt online threats
OpenAI’s researchers said they’re determined to continue identifying, preventing, and exposing online threats, noting that AI companies have a unique vantage point when it comes to online threats. It’s important to share findings with upstream and downstream providers as well as open-source researchers, they said, adding that “AI-powered investigative capabilities that flow from OpenAI’s innovations provide valuable tools to help protect democratic AI against the measures of adversarial authoritarian regimes.”