Narrow and General AI: Why the Yardstick Misleads Us

In May 1997 an IBM machine beat the world champion Garry Kasparov in a six-game rematch by 3.5 to 2.5, the first system to defeat a reigning world chess champion at standard tournament time controls. A machine built for one game. Is this the intelligence we mean when we speak of "artificial intelligence"?

What Separates Narrow from General?

Narrow intelligence masters a task or a class of tasks: translation, image classification, playing a game. General intelligence is the ability to learn new tasks and transfer experience from one field to another, as a child does while growing up.

The dream of the general kind is old in the field. In the Dartmouth workshop proposal (31 August 1955), its authors conjectured that "every aspect of learning or any other feature of intelligence can in principle be so precisely described" that a machine could simulate it, and asked for a two-month study by ten researchers in the summer of 1956. The matter was not settled that summer, nor for decades after.

Three Reasons the Yardstick Misleads

First: the exam is not the world. OpenAI announced that GPT-4 reached the 90th percentile on the Uniform Bar Exam. Eric Martínez re-estimated the result and published it in 2024, finding performance at about the 69th percentile among all test-takers in the July 2022 sitting, and about the 48th among first-time takers. The difference shows that the choice of comparison group can change the picture, and that an eye-catching number may be a product of measurement rather than ability.

Second: rapid progress is not generality. Stanford's AI Index (2025 edition) records jumps in a single year: about 18.8 points on the MMMU benchmark, 48.9 on GPQA, and the share of programming problems solved on SWE-bench rose from 4.4% to 71.7%. The same report says that models still struggle on more demanding reasoning tests such as PlanBench, and often fail at logic tasks that have a verifiably correct answer.

Third: skill is not intelligence. François Chollet argues that measuring performance on particular tasks is not enough, because skill reflects prior knowledge and training data as much as the system's capacity to generalise. He proposes defining intelligence as "skill-acquisition efficiency", and designed the ARC test for the purpose.

Lessons from History

After the Dartmouth proposal, promises were followed by disappointments. In 1973 James Lighthill wrote a report for Britain's Science Research Council saying that "in no part of the field have discoveries made so far produced the major impact that was then promised", and funding declined in a period dated 1974 to 1980 and known as the "first AI winter". This does not predict another winter; it is a lesson in the gap between promise and result.

So Where Do We Stand?

Researchers do not agree. Language models are far broader than Deep Blue and perform varied tasks without retraining, yet the evidence above shows them stumbling at planning and logic. They are closer to a very broad narrowness than to a complete general intelligence, and this remains a description, not a final verdict. For more, see the articles "Three Myths About Artificial Intelligence: Fluency, General Intelligence and Neutrality" and "How Do Large Language Models (LLMs) Like ChatGPT Work?".

What to Do

• Choose a specific task with a clear measure, then test performance on your own data and cases, not on a published score.
• Keep a human reviewer where mistakes are costly.
• Mind the quality of Arabic data; a tool is only as good as what it was fed.
• When someone promises you a date for general intelligence, ask for their definition and their yardstick.

The real value today lies in mastering useful narrow AI: in medicine, education and services. We take the means and work with what is in our hands, and we leave knowledge of the unseen to God.