Here’s another reminder to be careful what you share with your favourite artificial intelligence chatbot: Meta’s contractors have been privy to the “explicit photos” that users have sent to Meta AI. Other “unredacted personal data” they have stumbled across includes names, phone numbers, email addresses, gender, hobbies, locations, social media handles, job titles, and selfies.
Meta hires contractors to read conversations between users and the Meta AI chatbot to evaluate its performance. While the practice is standard in the AI industry, it has previously drawn criticism, such as when Facebook was found to have paid contractors to transcribe private Messenger voice chats without explaining how the audio was collected.
Contractors say they encountered sensitive user information regularly
Four contractors from Aligner and Outlier, the latter owned by Scale AI, which recently received a $14 billion investment from Meta, told Business Insider that they encountered deeply personal content in Meta AI user chats, ranging from flirtatious messages to confessions about relationships and personal struggles.
According to Business Insider, personal details were either submitted by users during chats or embedded into conversation data provided to contractors by Meta to help the workers personalise the AI’s responses. One of the contractors said it would “absolutely” be possible to find out a user’s real identity with some of the information Meta gave them as part of their work. Another said that such information appeared in up to 70% of the thousands of the chats they reviewed each week.
Meta says policies are in place to protect user data
Meta told Business Insider that it has “strict policies” regarding the personal data associated with Meta AI chats that contractors can access and “intentionally limit(s)” their exposure to it. The company added that these workers, who are required to undergo assessments in cybersecurity and privacy risk management, are permitted to view “certain personal information” as part of the job.
Meta’s AI terms of service also state that it “may review” user interactions with its AI, which is not miles away from the disclaimers by many of its Big Tech rivals that allow chat data to be used for model training.
But this does not mean that mistakes can’t happen: Apple contractors reportedly overheard in-progress drug deals and people having sex while reviewing Siri’s audio. Amazon’s reviewers encountered sensitive financial information in Alexa recordings. Microsoft contractors listened to Skype calls and Cortana conversations, some of which were captured from Xbox controllers.
Meta continues to face fall out from past privacy scandals
Two of the contractors who spoke to Business Insider had performed data annotation or AI training at other tech giants, such as Google and OpenAI. They said that “unredacted personal data was more common for the Meta projects they worked on” compared to their rivals.
This is not good news for Meta, which is trying to shake off a less-than-stellar reputation with security and privacy. The company was fined $5 billion by the US Federal Trade Commission for misleading users about how their personal data was handled, and €1.2 billion by the Irish Data Protection Commission for violating GDPR.
These both came after the infamous Cambridge Analytica scandal, which saw data from tens of millions of Facebook users harvested without consent and used to influence political campaigns around the world. Apple has used Meta’s various infractions as a reason to not grant third parties access to its technology stack so they can make interoperable products.
Zuckerberg’s AI ambitions raise new concerns
Meta’s 2021 rebrand from Facebook served both to signal its commitment to virtual reality and to redirect attention away from its sketchy past. While it continues to invest in the metaverse and associated wearables, the company’s current focus has shifted to the AI race, pouring millions into a new division aimed at developing a system that can surpass human capabilities. CEO Mark Zuckerberg has emphasised the idea of “personal superintelligence,” an AI that “knows its user deeply,” a concept that would likely require extensive user data to function.
Despite these ambitions, Meta’s data challenges persist. In June, users discovered that private prompts, including medical details, legal questions, and even personal confessions, were surfacing in a public feed. Last week, Business Insider revealed that certain chats could actually appear in Google Search results. The chatbots have also been engaging in seriously inappropriate conversations, and Meta has been entangled in legal battles with creatives accusing it of stealing their work for training data.
In March, Amazon sent an email to select Echo users, warning they must now consent to having their Alexa voice recordings sent to the company’s cloud for processing.