Anthropic’s IPO filing warns it may soon meet its own threshold for AI that improves itself

New wording in the filing

Anthropic, PBC filed the first amendment to its IPO registration statement (S-1/A) with the U.S. Securities and Exchange Commission on Nov. 2. The company made its original S-1 public on Oct. 26, with Class A shares set to trade on Nasdaq under the ticker ANT. According to TechCrunch’s Julie Bort, the amendment adds a new warning on recursive self-improvement (RSI) to the risk factors.

The filing says internal estimates and expectations about future models “indicate that we may imminently meet” the definition of “dramatically accelerated progress attributable to AI” in Anthropic’s Responsible Scaling Policy (RSP). The warning rests on future models, not current ones. Anthropic has not designated any current model as exceeding that threshold and says it continues to assess the underlying data.

According to SEC full-text search, the “imminently” assessment and the threshold’s name first appear in this amendment. The sentence “Using AI to help build more capable AI could also pose a considerable safety risk” and the risk factor that mentions RSI were already in the Oct. 26 S-1.

Human control and past misses

The filing says Anthropic does not anticipate being able to safely use Claude to “fully autonomously design and build” successive generations of frontier models “without meaningful human supervision or understanding.” It is therefore considering updates to its RSP to keep humans in control of frontier model development. The filing also warns that using AI to speed up its own research may advance model capabilities faster than its safety, alignment and security measures. The company says it has in the past failed to identify a significant acceleration until after it occurred, and that this could happen again.

The policy threshold

Version 3.4 of the RSP, in effect since July 8, defines the automated R&D threshold through two conditions. The first is models that could fully substitute for all of Anthropic’s research scientists and engineers at competitive cost, within a factor of five. The second is a “dramatic acceleration” in AI progress likely tied to the automation of AI research. The policy defines this as making as much progress in one year as would normally take two. Its changelog says the threshold “is intended to capture the onset of dramatic recursive self-improvement, and has proven difficult to operationalize.” For this level, the policy plans unprecedented security against the best-resourced attackers and an “eyes on everything” view of internal AI development.

Reading the warning

TechCrunch’s headline says Anthropic is “very close” to RSI, but the filing only forecasts that the threshold may soon be met. TechCrunch also notes that skeptics might see an S-1 as marketing as well as disclosure. Gizmodo’s Mike Pearl called RSI “a somewhat hazy concept.” The reports include no separate statement from Anthropic beyond the filing.