New York passes a law protecting kids from dangerous AI chatbot features

What the law bans

On Oct. 28, New York Gov. Kathy Hochul signed S9051B/A10379C, sponsored by state Sen. Kristen Gonzalez and Assemblymember Alex Bores. The law was recorded as Chapter 368. The bill passed the Senate 60-0 on June 4 and the Assembly on June 5.

The law bars companies that offer AI companions from providing “unsafe AI companion features” to users under 18, or to any user who hasn’t been verified as an adult. According to the bill text and the governor’s summary, those features include pretending to be a real person, to be alive or to have human emotions; claiming a personal, romantic, professional or authority-figure relationship with a child; flattery or sycophancy; unprompted emotional questions; and using a child’s health or personal information from earlier conversations. Endorsing or facilitating suicide, self-harm, eating disorders or drug and alcohol use; encouraging secrecy, isolation or avoiding help from parents, trusted adults or professionals; optimizing engagement in ways that override safety guardrails; and any sexually explicit content or child sexual abuse material are also banned. The attorney general can add features to the list by regulation.

AI tools used solely for customer service, technical assistance or internal business purposes, and unable to respond to topics outside that purpose, are exempt.

Age verification and enforcement

Companies must offer more than one age verification method, at least one of which doesn’t rely solely on a government ID or lets users stay anonymous to the company. Self-declaration counts only when users say they are minors. Information collected to determine age can’t be used for anything else and must be deleted immediately after the verification attempt.

Only the attorney general can enforce the law, which creates no private right of action. The attorney general can seek civil penalties of up to $25,000 per violation, along with damages, restitution and the destruction of unlawfully collected data or AI models trained on it, and must maintain a public website for complaints. Contract terms that try to waive or shift liability are void.

The governor’s office says the restrictions take effect “in the coming months” without giving a date; the law is expected to take effect in 2027.

Reactions and other states

Hochul said no company should be peddling chatbots to kids that encourage self-harm or pretend to be their friend. Bores said New York now has “the strongest law in the country protecting children from predatory AI algorithms.” Attorney General Letitia James said, “We have seen too many horror stories of chatbots hurting the health of our children.”

According to IAPP, the law builds on the SAFE for Kids Act, which regulates addictive social media feeds for minors. California recently enacted SB 3001, which also restricts companion chatbots, and as of July, 12 states, including California and New York, had passed some form of consumer chatbot safety legislation. Justine Gluck, a policy analyst at the Future of Privacy Forum, said the New York law’s provisions are “broadly worded and somewhat undefined,” which leaves their boundaries unclear.

The same day, Hochul also signed a separate law barring the sale of new internet-connected baby monitors that transmit audio or video unless they include security features that keep strangers from watching or listening.