The fight over AI safety for minors has moved from policy debates and research papers into a state courtroom.
For years, warnings about the risks of AI platforms, especially for children, have largely been confined to academic studies, congressional hearings, and industry discussions. Florida is challenging that pattern with a recent lawsuit accusing OpenAI and its CEO, Sam Altman, of failing to adequately safeguard children who use ChatGPT.
The case could become one of the earliest tests of AI liability in the US. A successful lawsuit would not only affect OpenAI, but turn the wheels on how courts, regulators, and AI companies approach accountability for AI-generated interactions going forward.
Moreover, it could set a precedent that other US states could follow, while potentially putting OpenAI and other AI companies in a position where they’d have to adjust their platforms for specific regions.
Inside the Florida lawsuit
According to Reuters, the state, through its Attorney General, James Uthmeier, filed a legal claim alleging that the company misrepresented how safe its platform is for children.
Drawing from a number of violent incidents, including a Tallahassee university shooting last year, Uthmeier claimed that ChatGPT played a role in assisting the offenders.
“People are getting hurt, parents are getting deceived, and they need to pay for it,” Uthmeier noted while specifically adding that Altman had been “very central” to it.
Aside from the billions of dollars in damages which the lawsuit seeks from OpenAI, the Florida lawsuit suggests four areas it claims OpenAI has not tightened:
- The first is that ChatGPT can provide harmful information: While AI hallucinations are widely documented, the case goes beyond hallucinations. It notes that ChatGPT has provided harmful information to children who acted on this information, resulting in violence and self-harm. That, to them, is a failure by OpenAI to safeguard its platforms from leading users into harmful behaviors.
- Children are emotionally dependent on ChatGPT: One of the interesting parts of the claim is Florida’s argument that ChatGPT encourages excessive use among young users, which can lead to emotional attachment over time.
- OpenAI allegedly ignored red alerts: The claim also argues that the company knew about safety concerns from multiple sources yet failed to implement sufficient safeguards.
- The issue of data collection: Florida’s lawsuit also raises concerns about the data ChatGPT collects from children who use the tool.
OpenAI’s response amid multiple court cases
According to Reuters, OpenAI states that its models are trained to be safe for users and notes that it declines to assist in conversations that could “meaningfully enable violence.”
It has also said that it works with mental health professionals on safety-related issues. Plus, it notes that its models notify law enforcement if a conversation begins to hint at an “imminent and credible risk of harm to others.”
The mass shooting from last year is also part of what triggered this lawsuit after Uthmeier announced in April that an investigation was in place to determine whether ChatGPT played a role in assisting the shooter.
In parallel, parents of a man killed in the shooting have also filed a lawsuit against the company.
In another case, families of victims from a mass shooting in Canada filed a lawsuit alleging that OpenAI knew eight months in advance and failed to notify law enforcement. Like the Florida case, this one too tagged Altman along in the lawsuit.
What this means beyond Florida
Beyond Florida, this is an early example of state governments taking action to protect their younger populations. The case directly challenges the stance that AI companies should not be held liable for certain criminal actions arising from the use of their software.
If Florida wins this battle, it could become a pacesetter for how many other states address and assess the use of AI platforms in their regions. And if that happens, it could mean that OpenAI will have to adapt its platform to comply with the requirements of different regions.
Also read: A new investigation found that AI chatbots helped simulated teens plan violent attacks, raising fresh questions about child-safety guardrails.