What Has AI Actually Proved for Our Quality of Life?

When we hear that AI "improves quality of life", we almost believe it without asking: what exactly was measured, and by whom? Below are four cases whose origins can be checked, and beside each, what must not be claimed for it.

Medicine: a second pair of eyes on the mammogram

In a study published in Nature in 2020, a system was trained on mammograms and compared with doctors on data from the UK and the US. It reduced false positives by an absolute 5.7% in the US and 1.2% in the UK, and false negatives by 9.4% and 2.7% respectively. In a reader study involving six radiologists, the system exceeded their average by an absolute margin of 11.5% on the area under the curve (AUC). In a simulation of the British double-reading process, it maintained performance no worse than the doctors while cutting the second reader's workload by 88%.

The limit not to be forgotten: these are results on past records, and the authors themselves concluded that they support later clinical trials, not that they make such trials unnecessary. We return to the danger of over-reliance on the machine in our article "AI as a Trusted Servant, on One Condition".

Education: an AI tutor in a physics lesson

Researchers at Harvard ran a randomised controlled trial in the introductory physics course for life-science students (autumn 2023). Each student learned one topic with an AI tutor built on GPT-4 to a research-based instructional design, and another in a conventional active-learning class. Across the two weeks, the median post-test score was 4.5 for the AI-tutor condition against 3.5 for the other (from 233 enrolled students, 194 of whom were analysed), and half of the students spent 49 minutes or less with the AI tutor, against sixty minutes of learning time in class. Students also reported higher engagement and motivation.

The authors' reservations: one course at one university; introductory material aimed at understanding and application rather than complex reasoning; and a result that depends on a particular design (expert-written prompts and pre-written step-by-step solutions). The effect on long-term retention was not measured. The trial shows that a specific design helped in a specific context, not that any chatbot teaches better than a teacher.

Roads: signals that adapt to congestion

Carnegie Mellon University developed Surtrac, which sets traffic signals according to actual traffic rather than fixed cycles. Its trial began in 2012 at 12 busy intersections in Pittsburgh. According to a report published in July 2017, the team behind the system estimates that it cut travel time by 25%, the number of braking events by 30% and idling time at signals by more than 40%, and that installation costs about $20,000 per intersection.

The limit of this case is that the figures are "estimates by the team", as the report put it, and it did not describe the method of measurement. They are a promising indicator that deserves independent verification, not a settled fact.

Floods: a warning days ahead

Researchers from Google published in Nature in 2024 a model that forecasts river flow daily up to seven days ahead. It was trained and tested on 5,680 gauging stations worldwide and can work in basins with no monitoring stations. It outperformed the European GloFAS system at about 49% to 70% of stations, depending on the size of the event, and matched or exceeded it at about 65% to 76%. The reliability of its five-day forecasts was at least that of GloFAS's same-day forecasts for floods recurring once a year. The paper reported that it operates in more than 80 countries, free of charge and without registration, and that it raises forecast quality in Africa to a level approaching what GloFAS achieves in Europe.

The caveat is that its authors noted the comparison is not entirely in their model's favour: GloFAS was calibrated on some of the stations and periods used in the evaluation, whereas the model was tested wholly out of sample. That, if anything, increases confidence in its result.

What the cases share

The four cases share three features. The benefit is narrow in scope: a system for a task, not general intelligence. It is measurable, as far as the methodology allows. And it extends human capability rather than excluding it: the doctor, teacher, engineer and emergency official remain the decision-makers.

It is fitting to give thanks for a blessing when it is proved, and to take the means of verification before generalising. To say "this was shown in that context" is more honest and more useful than "AI improves our lives", because the first formulation points us to what we should test next in our own setting.