AI in the Classroom: When Does It Teach and When Does It Weaken?

In a field experiment with nearly a thousand secondary-school students, researchers gave one group a general-purpose chatbot running on GPT-4 while they solved mathematics exercises. Their practice grades rose by about 48%. Then the tool was withdrawn and the test came, and they scored 17% below students who had never used it (Bastani et al., 2025).

The illusion of mastery

The researchers explained the result by saying the students made the tool a crutch: they handed over the thinking during practice and so built nothing to carry into the test. But the experiment included a second group that used an automated 'tutor' with teaching instructions designed to protect learning, that is, one that hints and asks questions rather than handing over the solution. Practice performance in that group rose by 127%, and these safeguards largely softened the decline on the test. The same tool, then, produced opposite outcomes depending on how it was designed. The study's title says as much: generative AI 'without guardrails' can harm learning.

The old dream: a tutor for every student

This promise has deep roots. In 1984 Benjamin Bloom presented results in which he saw that a student receiving one-to-one tutoring with mastery learning outperforms a conventional-class student by about two standard deviations, so that the average tutored student exceeded around 98% of the control class. Bloom thought such tutoring too costly for most societies to bear on a large scale, and called the challenge 'the 2 sigma problem'. Yet this famous result rests on two university dissertations, so it should not be treated as a final figure. The point is that the dream of AI in education is an old one: to bring individual teaching within reach of every student at reasonable cost.

When design is done well

In autumn 2023 a Harvard team ran a randomised crossover trial with 194 students in an introductory physics course; each student learned one lesson with an AI tutor at home and another through active learning in the classroom. The AI tutor was built on well-studied teaching principles and supplied with pre-written step-by-step solutions. The median learning gain in the AI group was more than double that of the classroom group (Kestin et al., 2025).

The result should be read with caution, as its authors themselves stressed: it comes from a single course at a single university, with introductory content; the AI tutor was designed by the study team after months of work; and the students' test results did not affect their grades. The authors said they do not presume that structured AI tutoring will outperform active learning in every context. The lesson is that careful pedagogical design made the difference, not the mere presence of a language model.

What remains for the teacher?

What the tool cannot do. UNESCO's 2023 guidance stresses a human-centred approach that takes the learner's age into account, and proposes setting an age limit for independent conversations with these platforms. The teacher reads the face of a frustrated pupil, knows when to leave the difficulty alone because understanding is born from it, and when to extend a hand. Not every struggle is harmful; working a problem through oneself is exactly what the tool weakens when it stands in for the student. Our article 'The End of the Rote-Learning Era?' discussed how the school's role changes in general; here we focus on the question of classroom use.

What to do in the classroom

• Tools that hint rather than solve: choose or configure the tool to question and guide the student before revealing the answer.
• Assessment that measures the route: when the tool writes the essay in seconds, a memorisation test loses its meaning; ask students to show their reasoning and to explain their decisions aloud.
• Teacher training in using and configuring the tool before rolling it out to students.
• Protecting minors' data: a child is not raw material for data collection; privacy is treated in our article 'Privacy in the Age of AI'.
• Checking the effect by measurement: try it in one class, then measure students' performance without the tool, before expanding.

The tool can make a student better able to think, or it can think in his place. The choice between the two is in our hands: in how we design, what we measure, and when we leave the student the honour of trying. By taking the means and relying upon God, we hope what we build for our children will support their minds, not replace them.