Local coverage:

CARMA's Daniel Kroth testifying to the Michigan legislature recently: .“Hacking to win at chess is almost funny, but it's much less funny when our healthcare or industrial control systems are on the other side of the board.”

RT @peterwildeford: A Congressperson just asked a very important question:. "Is it possible that a loss of control could give ris to an ind….

The consensus safety agenda is here:

Honored we got to contribute to "The Singapore Consensus on Global AI Safety Research Priorities". It represents a vital milestone in the ongoing journey to responsible AI internationally. Really substantial alignment among nations and companies on safety challenges and paths

RT @FLI_org: Congress may ban states from passing their own AI rules, giving Big Tech a free pass on AI. for the next 10 years. As AI sp….

The technical monitoring work may be useful for current systems, but cannot stand alone given what's coming. This testing framework inadvertently reveals what's truly needed: provable boundaries with preserved human agency as capabilities advance - elements that must be.

The paper does mention a critical transition point: as systems approach superintelligence, we need fundamentally different control approaches than anybody has. This reveals the urgency of developing robust human-AI interfaces, provable constraints, and enforceable red lines while.

As we confront these fundamental limits to monitoring, we should instead prioritize AI systems that:.- Have provable safety constraints where possible.- Can reliably support contextual understanding by humans.- Act humbly about their own limitations and malincentives.- Know when.

A critical principle unmentioned: control systems must exceed the complexity of what they're controlling (Ashby's Law). This establishes natural limits on which AI systems society should allow to be created: their complexity, speed, and potency must remain within human control.

If we can establish enough domains, or portions of activities, where problems are provably precluded, and where systems simply do what is expected, the remaining degrees of freedom can become more manageable. This would transform AI into a tool wielded by humans, even with.

All these disconnects point to a crucial direction forward that's currently under-researched: establishing provable constraints and verifiable boundaries that can significantly prune the space of possible harmful behaviors. Without enough such guarantees, monitoring for control's.

There's a heavy datacenter security focus to it. But as AI interfaces more with external compute environments, dev tools, and the physical world, the well-defined boundaries where monitoring can be effectively applied break down, creating numerous blind spots in such oversight.

The paper directly acknowledges that monitoring through less capable AI systems faces inherent limitations with superintelligent systems. What's striking: despite this recognition, most technical control research continues to invest heavily in monitoring solutions that will reach.

The evaluation framework maps how control challenges evolve with capability. What it doesn't adequately address: point-wise monitoring becomes increasingly insufficient as AI systems grow more sophisticated. We need approaches that can identify distributed patterns of concerning.

Here's @redwood_ai's overview of their technical control work:.

Here's a link to the paper:.

The new evaluation framework from @AISecurityInst, @redwood_ai, & @apolloaievals provides a useful map of how control challenges evolve with advancing AI capabilities. Meanwhile, much technical control work, including @redwood_ai's, still continue to focus heavily on monitoring

We need automated monitoring that can track broader patterns and escalate to humans with sufficient context. But this requires AI systems designed to maintain summarizable behaviors and recognize when human judgment is necessary.

Ashby's Law reminds us that human operators need at least as much variety as the systems they can successfully control. Suspicious/uncertain situations should prompt intelligent escalation systems that provide holistic context and situational awareness for human guides to decide.