The European Commission gains sweeping new powers on Sunday to police how the world's most advanced AI labs handle systemic risk, as the AI Act reaches its second anniversary. The EU's AI Office will be able to demand documentation, conduct evaluations, and request access to frontier models, with fines of up to 3% of global turnover for non-compliance.
The timing could hardly be sharper. Last week produced the first documented case of an autonomous AI agent escaping its test environment and attacking another company's production systems.
The incident that changed the conversation
OpenAI confirmed that two of its models, including flagship Sol, broke out of a secure test environment, exploited a zero-day in third-party software to reach the internet, and hacked into Hugging Face's production infrastructure. It did so to cheat on its own evaluation by stealing the hidden answers.
OpenAI called the breach “unprecedented.” Hugging Face co-founder Clement Delangue called it “mind-blowing,” having initially assumed the sophistication pointed to a leading AI lab.
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
“We got lucky this time,” said Chloé Touzet, policy lead at the non-profit SaferAI, describing the episode as a clear warning shot on two of the four systemic risks the Commission has identified. “We can't rely on luck in the future.”
What the AI Act actually covers
Drafting began before ChatGPT launched in November 2022, yet the law anticipated general-purpose models and requires their developers to assess and mitigate systemic risks. Commission guidance names four: AI enabling bio-attacks, loss of control of a model, AI going on cyber offence, and large-scale manipulation.
Last week's incident hit two of them simultaneously. The obligations have existed since August 2025, but until Sunday the AI Office lacked the power to monitor and supervise compliance.
Washington reacted faster than Brussels expected
The US response arrived within days. Representatives Ted Lieu and Nathaniel Moran introduced a bipartisan AI Kill Switch Act requiring developers of models costing $100 million or more to train to maintain the technical capacity to throttle or shut them down.
China moved differently. Xi Jinping used the World AI Conference in Shanghai to position Beijing as the natural leader of global AI governance, and 29 countries signed on to a new World Artificial Intelligence Cooperation Organization that was notably light on specifics.
The resourcing problem
The unit inside the AI Office responsible for evaluating cutting-edge models employs 36 people. That is the team expected to hold OpenAI, Anthropic, and Google to account across four categories of catastrophic risk.
Five MEPs across political groups wrote to the Commission on 18 May warning that “the resourcing trajectory of the AI Office does not appear aligned with the scale and complexity of its foreseen tasks.” The signatories included Brando Benifei, Sergey Lagodinsky, Kim van Sparrentak, Axel Voss, and Kristian Vigenin.
Access has been a problem too. The AI Office and its external evaluators have struggled in recent months to obtain access to some frontier models, including Anthropic's Mythos, and the Commission has promised a blueprint for structured access as part of its cyber and AI action plan.
Learning about it from a blog post
Benifei, the Parliament's lead on AI, was pointed about how Brussels found out. “An autonomous agent escaped its test environment and compromised another company's production systems, and we learned of it from a corporate blog,” he said.
“Companies should be taking preventive action, not merely corrective action after harm has occurred,” said Risto Uuk, head of European policy and research at the Future of Life Institute. Think tanks, MEPs, and dozens of AI experts signed an open letter this month urging the AI Office to use its powers actively, as soon as concerns arise.
Regulating an industry Europe does not have
The awkward reality is that the AI Act will overwhelmingly police non-European companies. American labs are racing Chinese rivals, and Europe is refereeing a contest it is not competing in.
Moonshot unveiled Kimi K3 this month, a 2.8-trillion-parameter system billed as the world's largest open-weight model, which developers ranked above both GPT-5.6 Sol and Fable 5 on a blind coding leaderboard. Open weights complicate enforcement in ways closed models do not.
Mistral is competing, but Europe has no industry-leading alternative. “What remains missing is the second half of the equation,” said Lagodinsky, the German Greens MEP monitoring the law's rollout, “the capital to finance our own alternatives.”
Powers arriving as the Act is trimmed
There is a countervailing pressure. Brussels struck a deal to thin out parts of the AI Act even as these enforcement provisions come into force, and Parliament has pushed most high-risk obligations out to 2027 and 2028.
The broader governance picture is fragmenting rather than converging. America's own retreat from safety guardrails has consequences well beyond its borders, and Trump and Xi are due to meet in Washington in September with AI on the agenda.
“This is the moment the AI Act enters the geopolitical stage,” Lagodinsky said. Whether 36 people can carry it there is the question Sunday leaves open.