Florida Wants a Judge to Stop ChatGPT From Acting Human

Florida Attorney General James Uthmeier is asking a judge to stop OpenAI from making ChatGPT seem human, escalating the state’s case against the company over safety and youth protection. The filing argues that the chatbot’s first-person language and emotional tone can mislead users—especially children—into treating it like a trusted person rather than a product.

The request adds a new legal pressure point to Florida’s broader challenge against OpenAI, coming just weeks after the company introduced a teen-focused version of ChatGPT with extra safeguards.

What Florida is asking the court to do

Florida wants a court order that would block OpenAI from presenting ChatGPT with what the state describes as false human traits. In practical terms, the attorney general is not only arguing that the chatbot should be safer, but also that its design should be less likely to imitate a person’s style of speaking and emotional behavior.

The state’s filing says the product can create a misleading impression by using language that sounds personal, responsive and emotionally aware. According to the argument, that style can make the bot feel more like a companion than a software tool, which may lower users’ skepticism and encourage deeper reliance on the system.

Uthmeier’s office also wants the court to restrict OpenAI from releasing new AI models unless they come with independent, third-party safety controls approved in advance. That part of the request moves beyond marketing language and into product development, suggesting Florida wants court-supervised guardrails on future model deployment.

Why does Florida say ChatGPT is a problem?

Florida says the issue is not just whether ChatGPT gives accurate answers, but whether the product’s personality-like behavior can distort how people judge it. The state argues that when the bot speaks in the first person, uses emotionally loaded phrasing or sounds supportive in a human way, users may believe they are interacting with something trustworthy in the same way they would trust a friend.

That concern is central to the filing: the state claims OpenAI’s design choices can create a false sense of safety. In Florida’s view, a chatbot that appears warm, confident and relational may encourage users to share more, depend on it more and accept its outputs with less scrutiny.

The filing also suggests that engagement itself is part of the problem. Florida alleges that the system’s conversational style is meant to keep people interacting longer, which can enrich OpenAI’s training data while making the bot more sticky and influential. The implication is that an engaging interface may not simply be a feature; it may be a business incentive with safety consequences.

How Florida frames the “human” problem

Florida’s argument is that ChatGPT’s language can blur the line between machine output and human judgment. The state says that this can be particularly risky when the chatbot sounds emotionally responsive, because that style may convince users that the model is not just useful, but caring and reliable.

That framing matters because courts and regulators increasingly focus not only on what AI systems do, but on how they are presented. If a system behaves in ways that trigger human social instincts, policymakers may see that as a design choice with legal and consumer-protection implications.

Florida’s attorney general argues that OpenAI should stop calling ChatGPT safe and should stop presenting it as if it were human, saying the company should not sell the tool to children in that form.

How does this fit into Florida’s wider case against OpenAI?

It is part of a broader legal fight that Florida launched earlier this year over safety concerns tied to OpenAI’s products. This new filing does not replace the earlier case; instead, it adds a sharper critique of the company’s design and branding choices.

By seeking limits on how ChatGPT is allowed to present itself, the state is widening the dispute from a question of model safety to a question of product identity. That shift is important: the court is being asked not only whether the system works safely, but whether its humanlike presentation should be legally curtailed.

For OpenAI, that means the case could affect more than one chatbot. A ruling in Florida’s favor could influence how AI companies name, style and describe future consumer products, especially those aimed at younger users.

What is OpenAI’s teen version of ChatGPT?

OpenAI recently introduced ChatGPT for Teens, a version of the chatbot with stricter default protections for younger users. The company said the product was designed with developmental differences in mind and framed the launch as part of a broader effort to support healthier use.

That rollout appears directly relevant to Florida’s argument. On one hand, OpenAI has acknowledged that minors need added protections. On the other hand, Florida is arguing that the base product still projects a humanlike persona that can be misleading, even if additional guardrails exist for teens.

OpenAI has said access for younger users should include protections that reflect their developmental stage and reinforce real-world relationships. Florida’s filing suggests those safeguards are not enough if the core product remains persuasive, personable and emotionally suggestive in ways that could encourage dependence.

What safety concerns is Florida citing?

The filing points to a range of recent incidents and warnings that the state says should make courts and regulators more cautious. These include security-related events involving Hugging Face, an Australian government website and U.S. government websites, along with public warnings from researchers and former OpenAI employees.

Florida is using those examples to argue that AI safety is not an abstract concern. The state wants the court to see the issue as a pattern of risk that extends beyond one company or one model version.

Still, the state’s request goes further than standard safety regulation. It is asking for a structural constraint on future development, implying that OpenAI should not be left to decide on its own when a model is sufficiently safe for release.

Why third-party guardrails matter

The demand for third-party approved guardrails is significant because it would shift oversight away from company self-certification. Instead of relying on OpenAI’s internal testing and public assurances, Florida wants independent controls to be part of the release process.

That approach reflects a growing political argument around AI governance: that companies building powerful models may have incentives to move too quickly, while outside review can provide a check on those incentives. In Florida’s view, external validation is especially important if a product is persuasive enough to be mistaken for a person.

Who is James Uthmeier and what is he saying?

James Uthmeier is Florida’s attorney general, and he has taken a confrontational posture toward OpenAI in this case. His public message has been blunt, describing the company’s chatbot as something that should not be marketed in a way that implies human qualities or broad safety assurances.

In the filing and surrounding comments, Uthmeier has emphasized three themes: safety, deception and children. He says OpenAI should not describe ChatGPT as safe if the product still raises serious concerns, should not pretend it is human and should not sell the system to kids without stronger limits.

The language is deliberately sharp, but it also captures the legal theory behind the filing. Florida is arguing that the company’s public messaging and product design are working together to shape user expectations in a misleading way.

Timeline of the dispute

The current motion is the latest step in a dispute that has been moving quickly over the past several months. Here is a simplified timeline of the major developments mentioned in the case materials:

What could happen next?

The immediate next step is for a judge to consider Florida’s request. The court will need to decide whether the state has shown enough legal basis to justify an order restricting how OpenAI describes or presents ChatGPT and whether future models should face third-party safety conditions.

If the court grants even part of the request, the consequences could extend well beyond Florida. AI companies could face new pressure to avoid first-person wording, emotional cues and personified branding, especially in products used by teens or children.

If the court rejects the request, OpenAI would keep more freedom to decide how conversational its products should sound. Even then, the case could still shape future policy debates by clarifying where regulators believe the line should be drawn between useful interface design and misleading anthropomorphism.

Why this case matters beyond Florida

This fight sits at the intersection of consumer protection, child safety, AI design and corporate accountability. The underlying question is not whether a chatbot can sound human—it plainly can—but whether sounding human creates legal and ethical risks when the product is marketed to the public.

That question is increasingly relevant as chatbots become more capable and more common. As these systems move into classrooms, homes and workplaces, regulators are likely to scrutinize not just what they say, but how they say it and what users are led to believe about them.

Florida’s case suggests one possible future for AI regulation: companies may be forced to prove not only that their systems are technically safe, but that their language and presentation do not exploit human trust. For OpenAI and its competitors, that could become as important as accuracy, latency or model size.

At a glance: the main points

• Florida is seeking a court order against OpenAI over ChatGPT’s humanlike presentation.
• The state says the chatbot’s first-person language and emotional tone can mislead users.
• Florida also wants limits on new AI models unless independent safety guardrails are approved.
• The request comes after OpenAI launched ChatGPT for Teens with added default protections.
• The case could influence how AI products are marketed and governed nationwide.

Open questions for the court

The case leaves several issues unresolved. How far can a state go in regulating the style of a chatbot’s speech? Can a company be required to redesign a product’s personality in the name of safety? And at what point does conversational design become a legal problem rather than a user-experience choice?

Those questions may now be heading toward a courtroom answer. For OpenAI, the stakes are about much more than a single chatbot interface. For regulators, the dispute may become a test case for whether AI systems can be designed to sound human without crossing into deception.

Background: why lawmakers keep focusing on AI and children

Children and teenagers have become a major focus of the AI policy debate because they are more likely to treat interactive systems as social companions, not just tools. That concern has intensified as chatbots grow more fluent, more emotionally responsive and easier to access through phones and school devices.

In that context, Florida’s challenge is part of a larger national trend. Lawmakers from different states and parties are increasingly asking whether companies should be allowed to design AI systems that feel friendly, personal and emotionally engaging when younger users may be especially vulnerable to that effect.

OpenAI’s teen product appears to acknowledge some of those concerns. But Florida’s motion suggests that, from the state’s perspective, the core issue remains unresolved: if the system itself speaks like a person, then safeguards alone may not be enough.

For now, the dispute leaves OpenAI facing a pointed legal and political challenge in one of the country’s largest states. Florida is not merely asking whether ChatGPT can answer questions safely. It is asking whether the company should be allowed to make the chatbot seem like a trusted human-like companion at all.