There is a great deal of discussion at the moment about whether AI development should be slowed, paced, governed or subjected to some form of international control.
Published: 26 October 2026
Authors: Oscar Tang, Sarah Reynolds & Alex Kirkhope
In September, the debate arrived once again at the UN. Frontier AI companies called for global standards. Political leaders discussed cooperation. Commentators discussed existential risk. Everyone discussed everyone else. It is difficult not to enjoy the theatre.
The cast is unusually strong. Sam Altman and Dario Amodei warn about the dangers of increasingly capable AI systems. Xi Jinping speaks of keeping AI under human control whilst pursuing technology leadership. In the US, there has been a tendency to frame AI as a geopolitical race and embrace the term "super intelligence". The world's most valuable technology companies insist that the technology is too important to regulate heavily, while simultaneously insisting that only they can be trusted to regulate it responsibly.
The underlying problem is straightforward. Global AI regulation requires global agreement. Those leading AI development, be it companies or countries, do not entirely agree on what they are trying to achieve.
The United States tends to see AI through the lens of competition. China views it as a question of national capability . Europe prefers its precautionary principle, safety and accountability, albeit with greater concern about competitiveness in the past year. In the UK meanwhile, considerable faith has been placed in the evaluation capabilities of the AI Security Institute (AISI), yet even its influence has been constrained by the reluctance of model developers and increasingly protectionist state players to grant access to the latest frontier models.
Everyone would like international standards, but only standards that reflect their own interests. Sure, common standards will emerge, reporting frameworks will be developed, and there will be a convergence around these standards and frameworks.
But businesses cannot wait for a grand bargain. The market almost certainly will not. For most organisations, their first AI regulator is not a government agency or quango, but a customer procurement team, an investor or an acquirer, asking uncomfortable questions about engineering and evidence.
Markets regulate faster than governments
The assumption sitting behind much of the public debate is that governance begins when legislators intervene. In practice, governance begins when counterparties start asking awkward questions.
Cybersecurity followed this trajectory.
Long before legislators arrived with EU NIS2 or DORA and a list of legal obligations, customers were sending security questionnaires, demanding independent assurance and refusing to contract with suppliers who could not satisfy expectations. ISO, BSI, ISC2, ISACA did the job. In other words, the market arrived first.
AI is already following the same path. Enterprise customers are rewriting procurement requirements. Investors are adding AI governance questions to commercial and legal diligence exercises, while boards expect management to explain where AI is deployed, and how associated risks are controlled. Businesses are being asked not merely whether they use AI, but whether they can evidence responsible use through inventories, impact assessments, testing records, incident procedures, contractual safeguards and governance artefacts. No treaty is required for any of this.
AI is both normal and novel
AI model developers and AI sceptics argue that our AI today is unlike anything humanity has ever seen. Those taking the opposite viewargue that AI is simply software. We think that this is beside the point.
AI is novel in the sense that capable AI systems exhibit behaviours that are difficult to predict, difficult to explain and difficult to control. These systems pose questions that previous generations of software did not. Many discussions focus exclusively on models. In practice, businesses deploy systems, and risks emerge not from a model in isolation, but from systems – from the interaction of models, users, business processes, automation, data flows and operational controls. Governance, therefore, is a whole-system exercise rather than a model exercise, and the frontier risk literature recognises this distinction. The relevant question is not whether AI will eventually become superintelligent, but whether current and emerging systems create observable risks that can be engineered against, today.
While these risks are often presented as entirely novel, many risks collapse into categories we already study, test and govern. An AI system that generates hallucinated outputs creates a quality problem, a compromised AI system creates a security problem, poor governance creates a management or reputational problem, and misuse of autonomous systems creates an oversight and control problem. An AI-enabled product that causes loss creates a liability problem.
There is a tendency in policy debates to speak as though every challenge requires an entirely new vocabulary, a new taxonomy, and a new branch of law or concept of liability. However, not every new technology creates a new problem – the reality is much less dramatic, and most organisations can already manage the AI risks that are most critical to them. The practical challenge is adapting those capabilities rather than reinventing them, and deciding what evidence should be required before increasingly capable systems are trusted in real environments.
Accountability is the real standard
The most interesting development is not the emergence of new legal duties, but of a common expectation around accountability.
The first generation of AI governance was focused on principles: fairness, transparency, explainability, safety, like the summit of a cascading policy hierarchy. We are in a time today where the real governance comes in the lower layer of procedures and guidelines. One lesson emerging from the frontier AI debate is that confidence in a model provider's safety claims is no longer sufficient. In private settings like commercial contracts, businesses increasingly want assurance. Who approved deployment? What testing occurred? What safeguards were implemented? What data was used? What happened during an incident? How is performance monitored after deployment and what happens if behaviour changes over time? These are not just compliance questions, but procurement, investment and transaction questions.
In other words, we’re moving from ideals to operations; from representations and warranties to assurance and verification; from what organisations say about AI to what they can prove. The vocabulary becomes dull, but the consequences become real, and the organisations that can demonstrate how systems were tested, controlled and monitored will be trusted more than those relying on policy statements alone.
Governments still matter
Private enforcement is powerful but incomplete. Contracts only bind the parties who sign them. Powerful customers can demand protections that smaller organisations cannot. Large market participants often succeed in defining standards for everyone else. Private arrangements do little for those outside those arrangements.
If an AI system causes loss to a third party, a procurement schedule is unlikely to provide comfort, and we move into tort law and insurance. Here, it is our governments and legislators who address systemic risks, establish minimum standards and protect those who lack leverage.
Where this goes next
Three things seem likely.
First, AI governance becomes a standard feature of commercial relationships. The question is not whether organisations use AI, but whether they can demonstrate responsible use when asked.
Second, accountability will become a transaction issue rather than compliance. Investors and buyers will care about dependencies on third-party models, training data provenance and quality, contractual allocation of risk and operational resilience. Good governance is a better valuation, and bad governance invites diligence and indemnity discussions.
Third, regulation will codify practices that emerged first in the market. This is how technology regulation frequently develops – new technologies generate uncertainty, customers create expectations, markets coalesce around standards, and the law, eventually, formalises them. We do not see the order reversing today.
Action, today
The public debate about AI governance focuses heavily on governments. That is understandable – governments write laws and journalists like theatre.
But organisations should pay at least as much attention to the commercial levers of the business – customers, investors, insurers, other counterparties – and how the market manages AI risk through engineering accountability. The dry, hard work of internal governance determines who wins contracts, secures investment and accesses markets.
The race to govern AI is often presented as a contest between nations but it’s increasingly a question of trust and negotiation between counterparties. Governments will continue to shape the legal baseline, but markets are already determining what good governance looks like. For boards, that means AI governance is a commercial driver rather than mere compliance. For procurement and legal, it is about supplier management. For investors and acquirers, it is core diligence. The legal risks are already here and the organisations that can evidence control, accountability and resilience will secure contracts, attract investment and earn trust. Those that cannot may find that the first meaningful AI regulator they encounter is not a government at all, but a customer.