Ten minutes with a chatbot was enough to change how people behaved once the chatbot disappeared. That is the central result of a paper by Grace Liu of Carnegie Mellon University, Brian Christian of UC Berkeley, Tsvetomira Dumbalska, Michiel Bakker and Rachit Dubey, first posted to arXiv on April 6, 2026 and presented at the Conference on Language Modeling in October. Across three randomized experiments with 1,222 recruited participants, people who had worked with an AI assistant solved fewer problems on their own afterward, and more of them skipped problems altogether.
Video: KTLA 5 news report on the study, from the KTLA 5 channel on YouTube. The figures in this article come from the paper and the university and press sources listed below.
How the experiments were built
Each experiment followed the same logic. A group of participants recruited on Prolific, all based in the United States, worked through a short run of problems with an AI assistant available, and a control group worked through the same problems without it. Then the assistant was taken away without warning, and everyone attempted a final set of problems alone. The final set was identical across conditions, which makes it the clean measure of independent ability.
The first experiment used fractions. Berkeley News describes a task of 15 basic fraction problems, where the AI group had ChatGPT open alongside and was free to ask it for the answer. The paper itself reports that GPT-5 served as the assistant in every experiment, that participants could use it however they liked with no penalty, and that they could skip any problem without losing anything. Sessions took roughly 10 to 15 minutes.
The second experiment added safeguards. It opened with three practice problems for everyone, with no AI and no hints, then ran 11 problems in easy, medium and hard blocks, then three final problems with neither the assistant nor the reference hints panel. The control group in this version received a how-to-solve-fractions hints panel and was told not to use a calculator. The third experiment moved from arithmetic to reading comprehension, with five questions answered with or without help and three more answered alone.
What the numbers show
The paper reports results for each experiment separately, and they are not equally strong. The list below gives the share of final problems solved and the share skipped, AI group first and control second, for participants whose data was analyzed.
- Experiment 1, fractions (307 analyzed: 185 AI, 122 control). Solve rate 0.57 against 0.73. Skip rate 0.20 against 0.11.
- Experiment 2, fractions (585 analyzed: 308 AI, 277 control). Solve rate 0.71 against 0.77. Skip rate 0.10 against 0.07, a difference the authors report as not statistically significant.
- Experiment 3, reading comprehension (168 analyzed: 85 AI, 83 control). Solve rate 0.76 against 0.89. Skip rate 0.08 against 0.01.
Recruited and analyzed counts differ because some participants were excluded; the press coverage quotes the recruited totals of 354, 667 and 201, which sum to 1,222. The solve-rate gaps were statistically significant in all three experiments, with the effect size largest in the first and third, where the AI group solved noticeably fewer problems alone. The smallest gap, in the second experiment, is the one produced by the design with the most safeguards, a point worth keeping in view when reading the headline claim.
During the assisted phase the pattern went the other way. Live Science reported that AI users solved more and gave up less while the assistant was available. Berkeley News described the same shape: the AI group started out more accurate, and its solve rate dropped sharply once the tool was removed, while the unassisted group held steady and finished stronger. The skip rate, the paper’s stand-in for giving up, moved in the opposite direction.
Who used the chatbot how
In the second experiment, participants reported how they had used the assistant. Sixty-one percent said they mostly asked for direct answers, 27 percent asked for hints or clarification, and 12 percent did not use it at all. The direct-answer users solved 0.65 of the final problems against 0.77 for controls, and skipped 0.13 against 0.07. Their solve rate fell by 0.10 from the pretest, while controls gained 0.01.
That split is the part of the study with the most practical meaning, and also the part the authors handle most carefully. The usage groups were not assigned; participants chose how to use the tool and described it afterward in a survey. The authors state that this analysis is correlational and self-reported, so it cannot show that asking for answers caused the decline, only that the decline was concentrated among people who did. One figure comparing hint-only users to controls could not be confirmed against the full text while this article was prepared, so it is left out here.
Reading comprehension, and why it matters
A reasonable objection to the fraction results is that arithmetic is a narrow skill, and that a person who stops trying on fractions has not necessarily lost anything that matters. The reading experiment addresses that directly. Comprehension draws on building a mental model of a passage, which is a different kind of work, and the same pattern appeared: a 0.76 solve rate against 0.89, and an 8 percent skip rate against 1 percent. Live Science summarized the result as suggesting that reduced persistence was not an artifact of the math problems but a general outcome of offloading thinking to the AI.
Liu acknowledged to Live Science that calculators also let people offload computation, which could produce similar declines. Her argument for why AI differs is breadth: a calculator replaces one operation, while a language model can take over reasoning across many domains at once. That is an argument rather than a measurement, and the study did not test calculators against chatbots.
The study arrived with a long public run-up. Berkeley News noted that a preliminary draft drew coverage from WIRED and Psychology Today in the spring, that the team presented the peer-reviewed version at the Conference on Language Modeling the week of October 9, and that the New York Times had published a story on the paper and related research a week earlier. The team spans Carnegie Mellon University, MIT, the University of Oxford, UCLA and UC Berkeley, according to the same article.
Productive struggle
Liu framed the stakes through what educators call desirable difficulties, the effortful parts of learning that build skill over time, according to Live Science. Her statement of the main result was that people who offload more show larger declines in persistence. Berkeley News gave the mechanism the team proposes: when answers arrive in seconds, anything slower starts to feel inefficient, and the experience of working through a problem, which the team calls productive struggle, goes missing. The paper makes the same argument in its abstract, saying persistence falls because AI conditions people to expect immediate answers.
This is a claim about expectations, and the experiments are consistent with it rather than proof of it. What they show directly is behavioral: the same people, facing the same final problems, gave up more often after the assistant was removed. Whether that changes anything that lasts beyond the session is the open question every limitation above points back to.
What the study cannot say
Several limits sit in the design. The sessions lasted about 10 to 15 minutes, so the findings describe a short exposure and a short test, not months of habitual use. The authors write that longer-term, naturalistic and longitudinal effects remain to be studied, and Live Science noted that scientists lack a full picture of how small instances of offloading affect lasting competence. Tracking the same people over a long period would be needed.
The first experiment may also have overstated the gap. The paper notes that its exclusion procedure could have let lower-ability participants into the AI condition, and the second experiment, with its pretest, was built to address that. The reduction in solve rate survived, but at a smaller size, and the skip-rate difference did not reach significance. Participants were paid online workers solving puzzle-like problems in a single sitting, a long way from a student with a deadline or an engineer with a codebase.
The paper is a preprint that was later peer reviewed for the conference. It has not been replicated elsewhere as far as the sources reviewed for this article show, and Live Science’s write-up quotes no independent experts, only the authors.
What the authors propose
The practical conclusion is aimed at the people who build the assistants. The abstract calls for AI development to prioritize scaffolding long-term competence alongside immediate task completion. Liu and her co-authors argue that a model should behave more like a tutor, giving directions and guidance toward a solution rather than the solution. Christian told Berkeley News that human abilities should be “augmented rather than supplanted”, and said the concern runs well beyond fractions and SAT passages to the way expertise is built in research and academia.
Nothing in the experiments tests whether a tutoring-style assistant would prevent the decline; that remains a hypothesis. It does line up with the one behavioral signal in the data, since the people who treated the assistant as an answer machine showed the largest drops. For readers curious about why chatbots default to handing over answers in the first place, the earlier explainer on how large language models like ChatGPT work covers how these systems are trained to produce a helpful-looking completion.
Sources
