We Cannot Govern What We Cannot Measure

On May 4, Carnegie Mellon will run live AI agent systems in front of 42 experts and see what breaks. The event, the second in a Brookings-CMU-Berkeley series, is structured around deployed systems — not panels, not white papers, not requests for comment. Participants will test evaluation frameworks against realistic failure cases and watch what happens. Whether the result is publishable findings or another research roadmap, nobody will say in advance.

The format is the story. After years of workshops — including a Brookings Institution recap cataloging what nobody can evaluate yet — the field is trying something different: live tests, real agents, documented failures. What changed is not new science. What changed is California.

AB 316, which took effect January 1, 2026, removed the autonomous operation defense. Companies can no longer claim their AI agent acted without human approval. Liability is now established in law before the measurement framework needed to assess it. What NIST publishes in 2026 will shape compliance checklists, vendor questionnaires, and courtroom arguments by 2027, according to Jones Walker LLP analysis. The legal question is answered. The technical question is not.

The NIST AI Agent Standards Initiative, the Commerce Department body working on those technical standards, launched its website in February 2026 with a goal of cementing U.S. dominance at the technological frontier. What the page does not describe is a working benchmark. NIST is collecting input. Nothing is finalized. A March 31 deadline for automated benchmark evaluation drafts had not yet passed when this was written.

NIST's own published documentation does include one concrete finding: multi-hop delegation — where one AI agent spawns another that calls a third — is flagged as an open question, not a solved problem. When agents operate across organizational boundaries this way, there is no agreed mechanism for tracking who authorized what and who is liable for the result. The OAuth standards that handle single-hop delegation, such as a customer service agent processing a refund, work today. The multi-hop case does not.

Enterprise organizations are not ignoring the problem. A survey of 306 AI agent practitioners conducted in 2025 found reliability issues as the top barrier to deployment. Deloitte's Emerging Technology research found 38 percent of organizations actively piloting agentic AI but only 14 percent with anything ready to deploy and just 11 percent running systems in production. The gap between pilot and production is not enthusiasm. It is confidence that the system will behave the same way tomorrow that it did today.

The performance gap has numbers. Research published by Kili Technology found a 37 percent drop in success rates when moving from a single controlled benchmark run to real-world conditions. Tasks that cleared 60 percent of the time on a single run dropped to 25 percent across eight consecutive runs.

NIST's own red-team research, conducted in January 2025 and cited by the Cloud Security Alliance, found that novel attack strategies against AI agents succeeded 81 percent of the time. The baseline rate was 11 percent. Defenses are not calibrated to what attackers can actually do.

Companies are making deployment decisions now: what to trust, what to disclose to counsel, what to roll out and what to wait on. The standards that would tell them whether those decisions are defensible do not exist. NIST is asking the right questions. The answers have not arrived yet.