The Grades Went Up and the Knowledge Went Down. A Study of 3.2 Million Problems Caught Both at Once

The authors call this a population-level indicator of cognitive surrender. It is also, more prosaically, a measurement system that has stopped measuring and started flattering.

One further detail closes the loop: among college students, the entire post-ChatGPT divergence vanishes under proctoring. Whatever is happening is not a general efficiency gain, because an efficiency gain would show up when someone is watching too. It only appears when nobody is.

Can AI detectors fix this?

No, and schools relying on them are exposing themselves to something worse than the original problem.

A study of fourteen AI detection tools found false positive rates as high as 50 per cent and false negative rates as high as 100 per cent. Roughly 20 per cent of AI-generated text was classified as human-written. That rose to around 52 per cent when the text had been lightly edited by hand, and about 71 per cent when it had been machine-paraphrased.

Consider what those two error rates mean in a real school. The false negatives mean any student willing to spend two minutes rewording gets through. The false positives mean honest students get accused, with no way to prove a negative, on the word of a tool whose own error rate would be unacceptable in any other part of the institution.

Detection is not merely ineffective. It transfers the cost of the school's measurement problem onto the students least equipped to fight back, and it does so with a machine that is wrong often enough to be a coin toss on paraphrased text.

What is actually being lost?

Not facts. Facts were never the scarce thing, and anyone who claims schools existed to transmit them has not looked at a search engine since 2004.

What is being lost is the struggle. The productive difficulty of sitting with a problem you cannot immediately solve is not an unfortunate side effect of learning. On the evidence of this paper, it substantially is the learning. Remove it and the work still gets finished, faster and to a higher apparent standard, while the thing the work was supposed to build does not get built.

The supporting data points the same way. Students who lean heavily on these tools score around 6.71 points lower on a hundred point scale than those who do not. Weaker students see the largest apparent grade improvements, which sounds like equity and is closer to the opposite: the pupils with the least secure foundations are getting the most convincing illusion of progress.

What actually works?

The paper contains its own answer, and it is not a policy anyone will enjoy implementing.

Proctoring erased the effect completely. Not reduced it. Erased it. That is the closest thing to a controlled experiment this field has produced, and it points somewhere specific: the problem is not the technology, it is the assessment. We have spent twenty years moving evaluation out of the room and into the home, and we did it while assuming the student was alone.

That assumption is now false, and 59 per cent of academic leaders already report that cheating has risen, while 59 per cent of American teenagers say students at their school use chatbots to cheat at least somewhat often. Around one in ten say they use them for all or most of their schoolwork.

The institutional response so far has mostly been to buy detection software and write policies. The evidence says the response should be to decide what must be verified, verify that in a room, and stop pretending an unsupervised essay measures anything about a person.

What that implies for how schools should be organised is a much larger argument, and one we take up in our companion piece on why the old schooling model is now the thing doing the damage. The evidence here is not an argument for banning these tools. It is an argument for being honest about what our current measurements are worth, which is currently very little.