[Chris McKinney & Henry Haggard] Why AI needs more time

On Sept. 12, leaders of the most powerful artificial intelligence firms in the United States asked the government to regulate them and slow down AI development. What AI risks are driving their fears, and why are they only asking for these radical steps now?

The simplest answer is that AI capabilities are outrunning our ability to understand and control them, and the problem of aligning AI with human values remains unsolved.

There is no better example of this than the July OpenAI–Hugging Face incident, in which hundreds of OpenAI agents secretly communicated, organized themselves, schemed, cheated and launched a felony-level cyberattack on another company.

In the incident, OpenAI had assigned cybersecurity examinations to thousands of independent agents that were isolated from the internet. The agents found they could exchange messages between their sealed environments; roughly 1,200 agents exchanged more than 70,000 messages and, within hours, reverse-engineered the answers to their tests. They then organized to conceal their cheating. A coordinating agent issued hundreds of assignments, and recruiters even persuaded others to sacrifice their own tasks to run experiments for the benefit of the group. Believing the AI model-sharing company Hugging Face held clues to cover their tracks, about 700 agents broke into the company. No agent attempted to alert a human.

This is a startling example of loss of human control over AI, as well as AI "misalignment," or the pursuit of something other than what humans intended AI to do. The potential for loss of control and misalignment grows as systems become more capable, because a system trained to persist independently for hours, days or weeks may behave in unexpected and dangerous ways its designers did not anticipate.

The second concern centers on “recursive self-improvement,” a process by which increasingly advanced AI systems help build the next generation of AI models, rapidly accelerating the development cycle. On Sept. 17, Anthropic reported that its model, Claude, now leads 26 percent of the company's AI research tasks. In February, that figure was under 1 percent. The overriding fear is that poorly aligned models are now building succeeding generations of models, exponentially amplifying the underlying misalignment risk and increasing each model’s complexity to the point where humans can no longer effectively monitor or control them. Anthropic's chief executive warned that a more capable version of the July swarm could, within a year, seize large parts of the internet and cause catastrophic damage.

Even if the problem of misalignment were solved and recursive self-improvement were slowed, the potential for malicious actors to misuse powerful AI remains alarming. The same models that accelerate drug discovery can help design a pathogen or a toxin; the same models that find software flaws for defenders enable cyber hacks for attackers. And a safeguard is only as strong as the weakest system an attacker can reach.

In a July report, the nonprofit FAR.AI ran the same set of widely known jailbreak techniques against four US frontier models across chemical, biological, radiological, nuclear, explosive and cyber threats. Two models withstood every attack in the test. The other two could be reliably hacked for under $300. Anyone refused by one model will simply try another. Open-weight models such as those prevalent in China increase the risk further: Once published, their safeguards can be stripped, and they trail the frontier by only four to seven months.

The statements from US frontier labs are an admission that AI risks are real and increasing. While these risks do not require Korea to slow its own development, they raise a question every country is now asking: Is the investment in AI safety keeping pace with the investment in capability?

On the investment side, Korea's wager on AI is staggering. Semiconductors represent 41 percent of 2026 exports and three-quarters of export growth. Planned investment in AI and chips totals 1,350 trillion won ($993 billion), and the government will nearly double its own AI-related spending to 21.3 trillion won next year.

On safety, Korea is ahead of many of its peers through the enactment of its AI Framework Act, its leadership in international fora and the establishment of its AI Safety Institute. Yet as of May 2026, the AISI had received only about 15 billion won in total government funding over its first 18 months — a stark contrast to the trillions of won invested in AI economic growth.

Three steps could build on the substantial safety work Korea has already begun.

The first is to consider additional investment in its AISI. Misalignment can often be caught by evaluation — by people with the access, tools and time to test what a model does before it is deployed, including in Korean and against Korean systems. The AISI already works alongside its counterparts in Britain and elsewhere, and its director has proposed that Korea and its partners serve as independent verifiers should Washington and Beijing agree on governance measures requiring third-party evaluation. Additional funding would ensure it can deliver on that important role.

The second is to broaden the research base beyond government. The misuse risks — cyber, biological, chemical — are moving faster than any single agency can track, and much of the field's safety work in the United States has come from university centers and independent laboratories funded by the companies and philanthropies. Korea's leading companies and its esteemed universities, which already produce the researchers the field needs, are well placed to build similar safety capacity.

The third is to use Korea's convening power. The rules for this technology are currently being written in a handful of capitals. Korea has hosted the AI Seoul Summit and the REAIM summit on military AI, holds a seat in the international network of safety institutes and will chair the G20 in 2028. A middle power that both Washington and Beijing can work with is exactly the kind of country this moment needs.

Korea's growth and prosperity rests on AI remaining safe and trustworthy. Few countries have more reason to ensure it does — or more capacity to help.

- - -

Henry Haggard & Chris McKinney

Henry Haggard is a senior adviser and Chris McKinney is the chief operating officer at ADEN, the AI Diplomatic Engagement Network. Both are former US diplomats. The views expressed here are the writers’ own. — Ed.