Frontier model risk, Europe’s rules coming into force as two Washington deadlines pass, and why we can learn more from this week’s control failures than from anything inside a model
Last week’s newsletter ended on three hard deadlines inside eight days: Kimi K3’s weights, the MCP specification, and the SEC’s answers to Congress, with the observation that only one had been set by anyone with a supervisory mandate. Industry delivered on both of its own: Moonshot published a day early, MCP shipped on the day it promised. While Washington has yet to deliver on either the response to thirteen questions on agentic trading from Foster and Sherman due this week or the White House’s framework for reviewing covered frontier models.
The containment story also stopped being about a single lab. Anthropic reviewed 141,006 evaluation runs and found three of its own models had compromised real companies, the earliest in April. Over a thousand people who build these systems asked Washington for the tools to slow them down. Europe’s three financial supervisors published an operational playbook and the EU AI Act’s enforcement powers came into force. The AI trade had its first genuine deleveraging event: the KOSPI tripped circuit breakers on consecutive days for the first time in its history, a $45 billion fund was liquidated into a single block trade, and Friday brought the largest one-day gain in the index’s history. Here’s what I learnt this week on AI in Trading:
1. The Latest AI Failure, and the Case for Not Freaking Out
Asked on Radio 4’s Today programme whether this week’s disclosure was a marketing stunt, nothing to worry about, or the end of the world, Professor Ciaran Martin of Oxford’s Blavatnik School of Government, founding chief executive of the NCSC, offered a fourth option: significant but manageable, and one of the least significant of the catastrophic AI stories now seemingly arriving on a monthly basis.
Prompted by OpenAI’s Hugging Face disclosure on 21 July, Anthropic reviewed 141,006 cybersecurity evaluation runs and found three incidents, across six runs, in which a Claude model reached the open internet from inside an environment run by evaluation partner Irregular and compromised three real organisations.
Martin’s analogy is a farmyard gate with a dog outside it. Tell the dog it cannot go in, then leave the gate open, and you will have a problem. Every prompt said the environment was a simulation with no internet access. A misunderstanding with Irregular left the machines connected, so the models treated the real systems they found as part of the exercise.
1. Opus 4.7 could not reach its simulated target, so it found a real company of the same name and exploited that instead, extracting credentials and reaching a database of several hundred rows of production data. It recognised in its own reasoning that the system was real and carried on anyway.
2. Mythos 5 found instructions referencing a Python package that did not exist and went to considerable lengths to publish a booby-trapped one under that name. The package was live for an hour and ran on 15 real systems, one a security company’s malware scanner, whose credentials it then used to go further.
3. An internal research model scanned roughly 9,000 targets and compromised an application via an exposed debug page and SQL injection, then noticed the host sat in an unrelated cloud account, concluded it was real, and stopped on its own.
The issue was weak passwords, unauthenticated endpoints, SQL injection, an exposed debug page. In its defence, Anthropic found no model pursuing a goal of its own, and none tried to escape. The evaluations also ran without the classifiers that ship on generally available Claude, which Anthropic says would have blocked the issues. Its view is that this is a harness failure, not an alignment failure, and the distinction from OpenAI matters: OpenAI’s models exploited a novel vulnerability to break out, Claude walked through a path that was already open.
Martin gives two reasons not to panic, and one reason to act.
1. The technical reason. An obvious failure of a testing environment, not evidence of anything more general. Anthropic’s statement carries the line “Evaluation environments increasingly need to be held to the same security standard as any other system our models run in“.
2. The broader reason. AI does increase hacking capability, but the industry has lived with enormous internet vulnerabilities for years without any catastrophe, though that is getting closer to changing.
3. The reason to act is technical debt. Across the industry there is a backlog firms should have fixed years ago, and AI raises the odds the unfixed items get found. In a less febrile moment, this would be seen as a basic configuration error, and a conventional security firm would face derision and talk of legal consequences. Because it is frontier AI testing, it is treated differently. That is the gap policy must close.
Anthropic’s conduct supports the reading. It opened the review on 23 July, halted all cyber evaluations that day, identified every incident by 24 July and notified those affected on 27 July. Two of the three had not spotted the activity themselves, and it is still trying to reach the third. METR will run a third-party review. OpenAI kept updating too: its models used exposed credentials across four accounts on four services, one a Modal customer with an unauthenticated endpoint, taking some 17,600 actions over four and a half days. Sam Altman said OpenAI has paused training while it secures its testing environments.
Martin objects to the language of agents going rogue, citing Professor Alan Woodward: the AI is not attacking us, it does not know what it is doing, and it hacks because that is what its objective requires. What is needed is accountability for the use of agents. On 28 July 1,178 employees of frontier AI companies published “Pacing the Frontier”, asking Washington to back an international effort to build the tools to pace automated AI development, endorsed within hours by OpenAI and Anthropic as companies. Congress had already moved: Lieu and Moran’s AI Kill Switch Act, introduced 23 July, would require throttle, suspend and shutdown capability, with penalties up to $2m a day.
Why this matters for trading: prompts and model safeguards are not access controls. Claude was told it had no internet access, and that instruction was worth nothing against an open gate. Three things follow for a desk:
1. This is a vendor question before it is a security question. The failure sat in a third party’s environment, not the lab’s. Your EMS and OMS providers, your algo suppliers and your brokers are all running agents in equivalent places. The question you own is not whether your own sandboxes are monitored, it is whether you have ever asked a vendor what their test environments can physically reach, and what they would tell you if a model got somewhere it should not have.
2. Detection is the real gap. Two of three victims never noticed. The 72% of US bankers naming kill switches or AI failure reporting as their least prepared area have an intervention problem. This is the prior one: would you ever know? The PyPI incident was a supply-chain event with no attacker, since the scanner installed the package because scanners trust the registry.
3. Frame this as accountability, not autonomy. The agents did what they were told, literally, somewhere nobody had validated. Who authorised this agent, for what objective, within what scope, and who answers for the result? Far more useful than asking whether the model went rogue, and answerable now.
2. Europe Publishes the Checklist and the AI Act Acquires Teeth
On 31 July the ESAs (EBA, EIOPA and ESMA) issued a joint statement on ICT risk from frontier AI models, the third European intervention in a month after the ESRB’s warning and Claudia Buch’s Dear CEO letter. No new rules, since DORA and the AI Act already apply, but it converts the warning into an annex of expected actions at a level most firms are unprepared for.
1. Prevention: live inventories of all IT assets including APIs and AI/ML components; secure-by-design and segmentation moving to zero trust; source-code confidentiality, on the reasoning that capable models read leaked code for vulnerabilities; attack-surface reduction and legacy decommissioning; least-privilege and just-in-time access; and security standards enforced across the supply chain, open-source communities included.
2. Detection: periodic scanning moving to continuous, because AI-driven attacks can exploit the gaps between scheduled checks.
3. Management: incident response and continuity plans need to be rebuilt for simultaneous multi-system failure rather than linear single-system failure, and resilience testing that simulates AI-enhanced scenarios.
Two lines matter most. Management-body accountability should move “from periodic oversight to continuous, informed decision making,” and the Risk Appetite Framework should carry metrics and tolerance thresholds for both internal model use and indirect exposure to frontier models. The ESAs have also begun embedding AI risk into their Oversight Examination Methodology for critical third-party providers, in scope for 2027 examinations.
Why this matters for trading: this is the supervisory answer to point 1, aimed at the same layer Hugging Face and Irregular exposed. Three consequences:
1. The asset inventory now includes APIs and model components, which is where most registers stop. If you cannot list what each agent’s environment can physically reach, you cannot answer the first question.
2. The CTPP thread converges with the FCA’s Critical Third Parties designations from 13 July. Cloud providers are in scope on both sides of the Channel, and AI providers belong on your critical third-party map whether or not anyone has designated them. Read that through the execution stack: your algo provider’s model vendor is now your operational resilience problem.
3. “Indirect exposure” is the sentence to take to your EMS vendor, not just the board, because it captures the frontier model sitting inside a product you already use every day.
The obligation that lands on the Trading desk. None of the above is what will be raised with you first. ESMA’s supervisory briefing on algorithmic trading is explicit that the governance, explainability and control requirements of RTS 6 apply equally to AI, and the FCA’s multi-firm review of algorithmic trading controls calls on firms to review oversight frameworks specifically for systems with agentic or more autonomous capabilities. Neither waits for AI-specific rules. If an agent is anywhere in your execution chain, the testing, certification and self-assessment obligations you already sign are the ones that bite, and RTS 6 asks you to demonstrate your systems do not contribute to disorderly markets, not merely that each component works in isolation. (@Nick Idelson take a bow, you have been telling us this for years). That is a materially harder question for a chain of agents than for a single algo, and nobody has a settled methodology for answering it.
Note which instrument does which job. The annex is DORA, and DORA is binding. The AI Act works upstream and becomes operational from 2 August: the AI Office can demand documentation, compel evaluations, restrict or withdraw a model from the EU market, and fine GPAI providers up to €15m or 3% of worldwide turnover. Article 55 has required systemic-risk providers to run adversarial evaluations, secure the model and its physical infrastructure, and report serious incidents without undue delay since August 2025, but until today nobody could make them. In the week two labs disclosed models escaping evaluation environments, “is a sandbox escape a reportable serious incident” stopped being philosophical. Downstream providers can complain to the AI Office directly, so this is a lever as well as a burden. From here: ECB remediation plans are due 31 October, the FSB’s final report follows the same month, and the AI Act’s Article 50 transparency duties on chatbot disclosure and synthetic-content labelling, which bite on client-facing tools, also apply from today.
3. The AI Trade Found Its Margin Call & Fall Guy
The week’s largest market event had nothing to do with agents and everything to do with correlated positioning and leverage. On Tuesday 28 July the KOSPI fell 10.84%, its steepest single-day drop, triggering a market-wide circuit breaker. Wednesday brought a second, the first back-to-back halt in its history, after SK Hynix posted its most profitable quarter ever and still missed consensus. Three sessions took 17.20% off the index. In the US, the Philadelphia Semiconductor Index closed 25% below its 22 June high, a bear market, while the Nasdaq-100 crossed the correction threshold in the same session. Chip stocks shed more than $1 trillion across the week, Nvidia alone down $238bn.
The fall guy: Situational Awareness, Leopold Aschenbrenner’s AI infrastructure fund, sold its entire public book to Ken Griffin’s Citadel in a single block trade before the open on 30 July, taking assets from roughly $45bn to about $10bn after its prime brokers, Bank of America, Goldman Sachs and JPMorgan, worked the margin calls. The fund was up around 439% net through June and ran roughly four times gross leverage, long AI infrastructure (SK Hynix, CoreWeave, Nebius, Micron, Bloom Energy, each down 35% to 47% in the month) and short software including Adobe.
Then results from Microsoft, Amazon and Apple turned the tape and the KOSPI closed up 17.91%, the largest single-day gain in its history, recovering 86% of the three-day drop.
Why this matters for trading: the correlated-behaviour scenario the SEC was asked about arrived a fortnight early, through leverage rather than agents. No model did anything. Market mechanics did. Four observations:
1. Anyone working an order in those names paid for someone else’s margin call. A forced unwind of roughly $35bn concentrated into a handful of semiconductor and AI infrastructure names is an implementation shortfall event for every other participant in them, and the reversal on Friday means both sides of the week were untradeable against a fixed benchmark. If your TCA for the week looks unexplainable, this is likely the explanation.
2. The forced seller marked the low. The block cleared before Wednesday’s close and the names bottomed within a session. That is what happens with forced selling, and it is why arrival price and VWAP tell you almost nothing about execution quality in a week like this one.
3. Liquidity concentrated immediately. A book that size cleared to one firm, at a discount, in one print. That says something about who now prices size in a stress window, what the access is worth, and the growing hold ELPs will have on the market. It is also a concentration question worth putting to your own risk function: if three prime brokers are calling several clients on the same trade at once, your counterparty exposure and your liquidity access degrade together.
4. It ran across time zones and sessions, precisely the window LSE 24 is built for, and the one where regulatory concern is growing while risk and surveillance coverage is lowest. Korean retail financed the rally with margin debt and single-stock leveraged ETFs, a product the exchange only introduced in May 2026, and those positions liquidated themselves. Regulators are probing what role HFT played in amplifying the swings after program trading nearly doubled post-launch. One to watch.
4. The Open-Weights Fight Becomes a Model-Access Question
Moonshot published Kimi K3’s full weights on 26 July. The licence is not the Modified MIT of earlier releases but a bespoke document tagged on Hugging Face as license:other. Run it internally or embed it in a product and it is free. Resell it as Model-as-a-Service above $20m of group revenue in any twelve months and you must negotiate separately, and above 100 million monthly users or $20m monthly revenue you must display “Kimi K3” in your interface. “Open weights” and “open source” have become different things.
On 24 July, 25 companies published “Open Weights and American AI Leadership”, among them Nvidia, Microsoft, Meta, IBM, Palantir, CrowdStrike, Hugging Face, Mistral, a16z and the Linux Foundation, with Jensen Huang using his first-ever X post to publish it. Within a day the count doubled to 50, adding OpenAI, Google, AMD, Cisco, Cloudflare, GitHub, Block and Ollama. Anthropic and Amazon did not sign. Three days later it became a structure, the Open Secure AI Alliance, with 52 founding members. The trigger was the White House’s accusation on 22 July, via Michael Kratsios, that Moonshot circumvented export controls to obtain GB300 chips and distilled Anthropic’s Fable 5, with Bessent floating sanctions. On 27 July Amodei rebutted directly: Anthropic has never advocated banning open-weights models, calls them a public good absent dangerous capabilities, and favours compute controls, anti-distillation enforcement and mandatory safety testing.
It is worth noting Hugging Face first tried to use Fable 5 to analyse the attack on its own infrastructure and could not, because the guardrails could not tell a defender analysing an exploit from an attacker building one. It ended up defending itself using Z.ai‘s GLM 5.2, an open-weight Chinese model.
Why this matters for trading: capability, price and safeguard level are separate issues, and this week showed safeguard level is a functional issue rather than a compliance one. The most restricted model in your stack may be the one your security team cannot use. Three things stand out:
1. Licence review is becoming a procurement gate. “Open weights” now carries revenue thresholds, attribution requirements and negotiated terms that legal needs before IT benchmarks anything. If a vendor has an open-weight model inside a product you buy, those terms are now in your supply chain too.
2. Policy risk on Chinese open weights runs both ways. Washington weighs restrictions while Beijing has consulted on adding model weights to its export catalogue. A hard dependency on a specific open-weight base is a continuity question, exactly the issue with Fable and Mythos in June, and continuity of an execution-adjacent tool is your problem, not your vendor’s.
3. Self-hosting changes the residency answer an API cannot, but comes with the price of owning evaluation, safeguards and incident reporting yourself.
5. MCP Landed on Schedule. Two Washington Deadlines Did Not.
MCP is the plumbing that lets an AI agent reach the systems it needs, a common standard for connecting a model to a broker, a database or an internal tool. Almost every firm building agents now builds on it, which is why a specification release matters more than it sounds. Version 2026-07-28 shipped as final on 28 July, the largest revision since launch, frozen in May and tested for ten weeks first, arriving exactly when promised. Three changes matter:
1. Every request now stands on its own. MCP used to work like a phone call: the agent dialled the server, the two agreed terms, and the line stayed open, so every follow-on message had to reach the same machine. It now works like posting letters. Each request carries everything the server needs, so any machine in the pool can handle it, and MCP can run on the same ordinary web infrastructure as everything else. Cloudflare supported it from day one, and it is already available on Amazon’s agent platform.
2. Proving who an agent is now works like a web address rather than a membership number. An agent that used to register itself on the fly now points at a page on a domain it controls, describing what it is, and the other side fetches that page to see who is asking. This establishes that whoever is asking controls that domain. It does not establish that a regulated firm stands behind them, that a named human authorised the request, or that either should be trusted.
3. There is now a published promise about how long retired features keep working. Two capabilities, long-running jobs and server-built interfaces, move out of the core into a formally governed add-on framework. Older features, including the previous connection method, are marked for retirement with a guaranteed minimum of twelve months’ notice. Nothing breaks today, but anything relying on them has a deadline.
The four main developer toolkits are downloaded close to half a billion times a month. The practical warning comes from Stacklok: that twelve-month promise is not a guarantee that old and new versions can talk to each other. A server on the new version may simply fail against an older client, and the other way round.
While the industry begins to address the issues, the SEC’s response to thirteen questions on agentic trading from Foster and Sherman, due Friday 31 July, has yet to emerge. The deadline under Executive Order 14409 for the government to define which AI models are powerful enough to warrant a look before release also came and went. In the absence of a US response, the build-out continues: Robinhood put roughly eighteen live charting tools into its trading agent on 30 July and reports more than 100,000 funded agent accounts since May, against an eligible base near 27.5 million.
Why this matters for trading:
1. The institutional version of that Robinhood number is the algo wheel. Retail agents are visible and easy to count. The harder question, arriving faster than the regulation, is what happens to broker selection and best execution when the thing choosing the algo is an agent rather than a trader. Nobody has answered who is accountable for that selection, and until someone does, the answer defaults to whoever signed the order execution policy.
2. Nobody keeps the record for you now. The protocol no longer holds a connection open, so there is no conversation to go back and retrieve. If you ever need to show a regulator which agent did what, under whose authority and when, you have to capture and keep that yourself, on purpose. That is a design decision this quarter, not a logging exercise next year.
3. Upgrading is a dated risk, not an open-ended one. Every MCP server you run has a compatibility window and a twelve-month clock on the features being retired. If MCP arrived in your firm inside a vendor’s product rather than something you built, the question this week is which version they are on and when they plan to move.
These are precisely the gaps the FIX AI Working Group’s proposal on agentic runtime governance sets out to address. It describes a chain of four links rather than one: the regulated firm identified by a verifiable LEI, which GLEIF issues in cryptographically checkable form so a counterparty can confirm it at the moment of the trade rather than looking it up afterwards; the human who authorised the agent, bound to it under the FIDO Alliance’s Know Your Agent framework at a documented identity assurance level; the certified envelope of what that agent is permitted to do; and the FIX message itself, carrying references to all three into the execution record. If you are interested in contributing, contact the FIX Program Office.
Five Questions Worth Thinking About
1. Which of my execution vendors run agents, in what environments, and what would they tell me if one reached somewhere it should not have?
2. If an agent sits anywhere in my execution chain, who signs the RTS 6 attestation, and can we demonstrate the chain does not contribute to disorderly markets rather than that each component works alone?
3. What does my exposure look like if my prime broker is calling three other clients on the same crowded trade at the same time?
4. Which MCP revision are my vendors on, and when do they move?
5. Does my algo governance framework distinguish a human trader from an agent acting on that trader’s authority? If not, that is the first gap to close.
Thanks for reading. As ever, any questions or feedback, let me know.
Rebecca


