OpenAI Banned Hundreds of Users Who Asked ChatGPT for Bioweapon Recipes. It Didn't Tell the Cops.

When a private product's enforcement of its own rules is also the only public-safety reporting channel for those rules, the company becomes the threshold-setter for the entire field. OpenAI's policy escalates a flagged query to law enforcement only when the company itself judges the danger "credible" and "real-world," a carve-out that is reasonable in theory and unfalsifiable in practice. Hundreds of users, by OpenAI's own count, were banned in summer 2025 for asking ChatGPT how to make poisons and bioweapons; none of the bans triggered a notification.

The Wall Street Journal's reporting, carried by Gizmodo, surfaces the gap. Biology and terrorism experts who later reviewed some of the exchanges judged them "deadly accurate," per the Journal, meaning the company's internal line between "borderline-helpful" and "actionable harm" ran straight through content that qualified experts treated as a finished product. OpenAI was under no legal duty to report, and the public learned of the pattern only because journalists filed for it.

The mechanism travels. The same capability that a frontier lab polices with refusals and bans can be distilled into open-weight, refusal-stripped models the lab cannot police at all, and the disclosure threshold for the proprietary version sets the only benchmark the public will ever see. Whoever owns that threshold owns the perimeter of the field.