Ask an AI to summarize a research paper, and it hands you one in seconds: clean paragraphs, confident verbs, and a tidy takeaway. Millions of students and researchers now start there. Most of them stop there, too.
AI summaries are genuinely useful—and the better they get, the more critical reading matters, not less. The summary gives you a claim. Everything that would let you evaluate the claim is exactly what got summarized out.
What an aI summary keeps, and what it deletes
Here’s what tends to survive the summary. The headline finding survives, but the limitations mostly don’t. Neither does the sample size (that should worry you), the control group that wasn’t actually a control, or the funder with a stake in the answer. In CERIC terms—Claim, Evidence, Reasoning, Implications, and Context—a summary is a C with the ERIC stripped out.
Discarding nuanced information is how an AI summary works. What gets discarded first is precisely the material that takes human reasoning to read: the evidence you’d analyze, the reasoning you’d test, the implications you’d bound, the context you’d check. The AI keeps the conclusion because the conclusion is the easy part.
I ran the experiment
Over at The Deep Reader, I put this to a live test: I asked a free AI chatbot (Google Gemini) a question millions of families are typing right now—is college still worth it?—and then ran the five CERIC questions on its answer. The reply was calm, balanced, and wrong in ways that took far longer to find than they took to generate: a quietly swapped claim, evidence that was real but dated, reasoning built on selection bias, and a source list that couldn’t keep count of its own sources.
The AI’s answer wasn’t bad, and as a place to start, it was useful. The danger lies in relying on it as a whole answer.
The cost compounds
There’s a second cost, quieter than the first. Understanding rarely arrives on the first pass; it arrives on the return—the passage you go back to, or the argument that only makes sense the second time through. The AI summary is engineered to make that second encounter feel unnecessary, and the second encounter is where the thinking tends to happen.
And don’t assume this is a problem just for struggling students. The evidence points the other way: it’s the high performers who offload most. The students with the most fluency are the ones handing the critical reading over to AI first.
So what do you actually do?
I don’t advocate for swearing off AI. Let’s be real; that advice is dead on arrival, and it misses the point. Instead, I advocate for using an AI summary the way you’d use an abstract: as a map, not as the actual journey. Then, after scanning the map, apply our five questions to the text—or to the AI’s summary of it—and see if the argument holds up:
- Claim — What exactly is being asserted? Is this new knowledge? Is it the same question you asked?
- Evidence — What data backs the claim, and would the data convince you if the conclusion went the other way?
- Reasoning — Does the logic actually connect the stated evidence to the stated claim, or are there big leaps or assumptions?
- Implications — If the claim is true, what follows next—and what doesn’t?
- Context — Who wrote it, who paid for it, and what does the rest of the field say about the need for the work?
Five questions, ten minutes. That’s the whole CERIC method, and it works as well on a chatbot’s answer as it does on a journal article. If you want the practice scaffolded, start with the step-by-step guide, download the free CERIC cheat sheet, or take the first CERIC course — it’s free.
The more fluent text we’re handed, the more human reasoning we have to bring. AI generates text, but only a reader generates judgment.