AI or ‘super intelligence’? Trump isn’t the only one who is confused
· New Scientist
In the decades since the field began, most of the debate surrounding artificial intelligence has been about the intelligence part: can machines really think like we do? But in recent weeks, US President Donald Trump has kicked off an argument about the other part – and joined a long tradition of shifting the goal posts when it comes to AI.
Visit freshyourfeel.org for more information.
Read morearXiv might not survive AI slop onslaught, warn mathematicians
“The use of the word artificial makes intelligence fake, it makes it sound fake, and it is not fake. It’s actually amazing,” said Trump during a speech at the UN General Assembly on 22 September. He is calling for artificial intelligence to be rebranded as “super intelligence”, backing this move with an executive order signed on 29 September and the launch this week of a body called the Super Intelligence Force for coordinating US AI policy.
Putting aside the fact that “Super Intelligence Force” sounds like a Saturday morning cartoon, what are we to make of this attempted rebrand? Trump isn’t actually alone in seeking to distance modern AI systems from the original conception of AI in the 1940s and 50s. Last year, Meta CEO Mark Zuckberg published a note declaring that “superintelligence” (one word) is now in sight. Reaching further back, the word was originally popularised by philosopher Nick Bostrom as the title of his 2014 book.
Advertisement
if( window?.adverts?.addToArray ) { window.adverts.addToArray( { "pos": "mid-article-slot" } ); }For me, Trump’s super intelligence and Bostrom’s superintelligence have distinct meanings that are worth preserving. Trump’s rebrand is an effort to cement US dominance in the field of AI – his executive order states that “the term ‘Super Intelligence’ more appropriately captures the promise, potential, and rapidly advancing capabilities of these technologies”. But superintelligence as envisioned by Bostrom goes far beyond the capabilities of our current AI systems. To understand why, we need to dig into history a bit more.
Long before the birth of ChatGPT, AI researchers distinguished between weak AI and strong AI, terms first defined by philosopher John Searle in a 1980 paper titled “Minds, brains, and programs”. Loosely speaking, he considered weak AI to be a computer simulation of a human mind, but not an actual mind itself, whereas he felt strong AI describes actual cognition emerging from software. Searle dismissed this latter idea as ridiculous: “No one supposes… that a computer simulation of a rainstorm will leave us all drenched. Why on earth would anyone suppose that a computer simulation of understanding actually understood anything?” Decades on, I find it hard to disagree.
In the wake of Searle’s influential paper, weak and strong AI underwent their own Trump-style rebranding, as people began to discuss differences in AI capabilities rather than AI consciousness – perhaps because the former is much easier to measure and moves the arena of debate from the philosophy department to the computer science lab. Weak AI merged with another term, narrow AI, to mean a system capable of human-level performance at a specific task. We have been living with narrow AI for decades, depending on how you want to define it, encompassing everything from spam filters to chess playing to driverless cars. In some sense, once something becomes achievable by narrow AI, it is just software – a phenomenon known as the AI effect.
Strong AI, meanwhile, gave way to general AI – a system capable of human-level performance at any task. A general AI is a highly desirable goal because it means that, rather than having to build a new system for every possible task, the AI would be able to transfer learning from one domain to another, just as a human can. In 2002, general AI morphed into “artificial general intelligence”, or AGI, largely due to Shane Legg (who would go on to co-found the firm DeepMind) popularising the term.
Most of the big AI companies, including DeepMind and OpenAI, were founded with the stated goal of achieving AGI. In the 2010s, this was seen as a slightly wacky and sci-fi research goal. Today, it is a business model, with trillions of dollars of investment riding on building an AGI in the near-future.
US President Donald Trump with tech executives after a meeting on AI safety at the White HouseTierney L Cross/POOL/EPA/ShutterstockDespite this, definitions of AGI are woolly. OpenAI’s charter defines it as “highly autonomous systems that outperform humans at most economically valuable work”. An agreement between OpenAI and Microsoft that was leaked in 2024 defines it as a system capable of generating $100 billion in profit. OpenAI isn’t yet profitable, with expenditure far exceeding revenue, but that didn’t prevent its president, Greg Brockman, from saying “welcome to the AGI era” with the release of its Astra model in September.
However you define AGI, it isn’t the same thing as superintelligence. Bostrom defines this plainly in his book as “any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest”. In other words, a superintelligence must be more cognitively capable than any human scientist, artist, philosopher or other cognitive performer that ever lived. It isn’t enough to be better than Albert Einstein or Marie Curie – you must also beat William Shakespeare, Leonardo da Vinci, Steve Jobs, Jane Austen and many, many others. Clearly, we haven’t achieved superintelligence. A real superintelligence would probably transform the world in ways we can’t imagine.
There is one problem running though all of these definitions, though, and it lies in the second part of AI: what actually is intelligence, and how do we measure different levels of it?
The standard answer is intelligence quotient, or IQ. Attempts to measure the IQ of various high-end AI models (by having them produce answers to IQ test questions) give ranges of 130 to 150 IQ, although this is complicated by the fact that the tests themselves may have formed part of the models’ training data. Human IQ scores have reached over 200, often in young children because the scores depend on age, but higher scores are increasingly difficult to measure reliably. More pressingly, IQ is now widely considered to be a flawed measure of intelligence, rooted in eugenics, that ignores many aspects of human capability. Attempting to apply it to AI feels fairly fruitless, then.
As an alternative, some computer scientists have tried to come up with AI-specific intelligence tests. One of these is ARC-AGI, which looks not only at the capabilities of a model, but also at the cost of running it, meaning that less computationally intensive models can score higher. The test is now in its third iteration, and one of OpenAI’s latest models, GPT-6.1 Sol, scores 96.2 per cent, completing the test at a reasonably cheap $3800. Does that make it AGI? I don’t think so, and I would expect a fourth version of the ARC-AGI test to come along soon, resetting the bar that models must exceed.
Does this mean we will never achieve AGI, and then superintelligence? Despite the certainty of the AI firms that we will – which is probably half real belief and half marketing – I think at this point, it is impossible to say. The meanings of these terms and our methods for evaluating them are constantly shifting. What seemed liked magic in 2022 – a computer that could answer you back – is now an annoyance in 2026, with Google’s AI Overviews often producing confidently incorrect information at the top of your search results.
Today, the most impressive achievements in AI seem to come from mathematics, but perhaps that will soon become passé. My feeling is that, if AGI ever comes, there won’t be one moment in which we declare it has arrived, but rather a gradual realisation that it is already here and we’ve just started calling it something else.
OpenAI mistranslated mathematics into code for its Navier-Stokes proofWhen OpenAI announced its surprise solution to the Navier-Stokes problem, it produced one proof for humans and one for computers – however, they don’t match
Further AI reading:
- A software developer has used Anthropic’s Opus 5.5 model to recreate Photoshop and other Adobe products, releasing the code for free on GitHub. The legal implications are unclear, and while it is possible to do this kind of reverse engineering in a legally acceptable way, I would be very surprised if Adobe’s lawyers don’t come knocking soon.
- Nolla Health is piloting “AI-powered prescriptions” for acne treatment in Utah, using an app that asks patients to fill in a questionnaire and scan their face. Initially, two licensed physicians will review the AI-generated prescription before the patient receives it. In further trials, such reviews will happen only after the prescription is already sent.
- Google has released Playground, an AI app that generates playable games from text prompts. It is available only in the US, so I haven’t used it myself, but my own experiments with using Claude to generate games have seen mixed results – try one here if you are curious.
- Staying on games, a new trend has seen people “merge” two different video games by using AI to recreate the code of one inside the other. This has led to the likes of Minecraft inside Elden Ring. It’s an amusing idea and a great viral video, though it isn’t clear how well these actually play – and again, legal questions loom large.
- ChatGPT has a new “Intelligent UI” feature that allows the chatbot to answer questions using interactive visualisations, rather than just text. In my own AI use, I’ve found that having the model create a temporary website I can poke at is often very helpful, and this essentially brings that capability to anyone who doesn’t want to set up their own web server.