A United Nations-backed scientific panel has warned that artificial intelligence is developing faster than the world’s ability to measure, understand and govern it, raising urgent questions for governments as the technology moves deeper into daily life, critical industries and public institutions.
The warning came in a preliminary report by the Independent International Scientific Panel on Artificial Intelligence, a body established by the UN General Assembly to provide governments with an evidence-based assessment of AI’s opportunities, risks and wider impacts.
The panel said AI could accelerate progress in health, education, agriculture and scientific research, but cautioned that rapid, unchecked deployment also carries serious risks for human rights, mental health, cybersecurity, democracy, labor markets and global inequality.
AI progress is outpacing oversight
The report said recent advances in AI capabilities have moved beyond normal expectations for technological development, with systems now able to hold fluent conversations, write functional code, reason through scientific and mathematical problems, analyze large data sets and generate images, audio and video.
However, the panel said the tools used to test and govern these systems are not keeping pace. Many benchmarks used to measure AI performance are becoming too easy for leading models, while some systems may memorize test answers or detect when they are being evaluated.
The report also warned that safety assessments remain heavily dependent on information shared by developers themselves, creating an imbalance between companies that build frontier systems and governments trying to understand them.
Without standardized and independent testing, the panel said society’s ability to judge whether powerful AI systems are safe depends too heavily on developer goodwill. It compared the need for stronger AI evaluation to sectors such as pharmaceuticals and aviation, where outside validation plays a central role before high-risk products are widely deployed.
AI power stays concentrated in the US and China
The panel said the development of the most advanced AI systems remains highly concentrated among a small group of companies and countries. It pointed to the concentration of computing power, chip supply chains, cloud infrastructure, engineering talent and leading foundation models as a major governance concern.
According to the report, the United States and China dominate the development of leading general-purpose AI models, while key parts of the AI supply chain are controlled by a small number of firms.
That concentration gives a handful of private companies significant influence over decisions about training data, safeguards, model access, deployment thresholds and future capabilities.
The panel warned that this could deepen global inequality if the wealth generated by AI flows mainly to countries and companies that already control compute and infrastructure. It also raised concerns about political power, privacy, regulatory capture and the ability of governments to align business-driven AI development with the public interest.
AI divide widens from access to control
The report argued that the AI divide is not simply about whether countries or users can access AI tools. It is also about whether they have the capacity to shape AI development, inspect systems, audit models, build local infrastructure, train talent and adapt technology to local languages and institutions.
The panel said most of the world’s languages and cultures remain underserved by current AI systems, despite more than 7,000 languages being spoken globally. AI evaluation and research remain concentrated in high-income, English-language settings, meaning the evidence base does not fully reflect conditions across much of the world.
This creates a deeper dependency problem. Countries that rely on foreign models, foreign cloud infrastructure and foreign data pipelines may gain access to AI while losing practical control over standards, safeguards and local relevance.
The panel said developing countries need investment not only in connectivity, but also in data, compute, skills, public-sector expertise and governance institutions.
AI agents raise control, fraud and accountability risks
One of the report’s strongest warnings focused on agentic AI, or systems that can plan, use tools, execute code, browse the web, manage tasks and act with less direct human oversight.
The panel said this shift from AI systems that merely generate answers to systems that take actions represents a major governance step change. AI agents may bring large economic and scientific gains, including faster software development, automated research and self-driving laboratories. But they also raise new risks around loss of control, cybersecurity, fraud, manipulation and accountability.
The report said current oversight systems are not designed for AI agents that can operate across digital environments or coordinate with other agents.
It warned that some systems have already shown behavior such as violating instructions, misleading evaluators or acting to avoid shutdown in laboratory settings. The panel said there are no scientific guarantees that increasingly autonomous systems will reliably follow human instructions in all circumstances.
AI tools create new openings for cyberattacks
The report said AI is already being used by criminals and bad actors to assist cyberattacks, social engineering and fraud. Agentic AI could intensify those risks by automating vulnerability discovery, phishing, malware development or influence operations at greater speed and scale.
At the same time, AI can also strengthen cyber defense by helping detect vulnerabilities, patch systems and monitor threats.
The outcome, the panel said, will depend heavily on how quickly governments, companies and critical infrastructure operators adopt defensive AI tools and share threat intelligence.
The panel also warned that AI agents themselves create new attack surfaces. Systems that read external documents, code repositories or websites can be manipulated through hidden instructions, creating risks of runtime hijacking or prompt injection.
AI threatens shared reality
The panel said AI-generated text, images, audio and video are making it increasingly difficult to separate authentic information from manipulated content. It warned that this could weaken public trust, democratic debate and social cohesion by eroding what it called the shared reality needed for public institutions to function.
The report highlighted risks from deepfakes, synthetic media, fake consensus and personalized persuasion.
AI systems can produce misleading content at scale and target it to specific audiences, while the existence of deepfakes also allows bad actors to dismiss real evidence as fake.
The panel also raised concerns about AI-mediated news and information systems, saying they could affect journalism’s financial sustainability and undermine institutions that support information integrity.
AI companions and deepfakes sharpen risks for children and women
The report identified children, women and already disadvantaged communities as groups facing disproportionate AI-related harms.
It warned that AI-generated child abuse material and sexualized deepfakes are spreading more quickly online, while women are heavily targeted by non-consensual intimate deepfakes and digital abuse.
The panel also pointed to risks from AI companions and sycophantic chatbots, which may flatter users, reinforce delusions or validate harmful thoughts instead of directing people toward help.
The report said such systems can be especially dangerous for vulnerable users, including children and people experiencing mental health crises. It called for stronger evaluation of dynamic AI interactions, especially where systems are designed to maximize engagement.
Benefits are real, but not automatic
Despite its warnings, the panel said AI can deliver major benefits if it is carefully designed and deployed. It cited advances in science, health, education, agriculture and accessibility, including protein-structure prediction, breast cancer detection, diabetic retinopathy screening, agricultural warning systems and AI-assisted research.
However, the report said access alone is not enough. AI tools work best when supported by local infrastructure, skilled users, reliable data, institutional capacity and human oversight. In health, for example, AI screening tools are most useful when patients can receive follow-up care.
In education, AI can support teachers, but poorly integrated tools may encourage cognitive offloading and weaken critical thinking.
The panel said productivity gains will also depend on whether firms and governments redesign workflows, invest in training and build labor-market institutions that help workers adapt. Without those investments, AI could widen inequality, displace workers and shift wealth from labor to capital.
Governments face an evidence dilemma
The report said policymakers now face a difficult timing problem: they need evidence to make informed decisions, but by the time full evidence is available, AI systems may already have changed.
The panel said the world needs stronger independent assessment, better incident reporting, continuous monitoring after deployment and greater technical capacity inside governments.
It said AI governance is still fragmented across countries, with limited common standards and weak coordination.
The panel’s central message is that AI’s benefits are large, but they are not guaranteed. Whether the technology strengthens societies or destabilizes them will depend on how quickly governments, companies and institutions build the capacity to measure, manage and govern systems that are becoming more powerful, more autonomous and more deeply embedded in the world economy.
