AI Coding Assistants and Developer Skill Loss: What the Research Actually Shows
A randomized trial found AI users scored 17 points lower on comprehension after learning a new library — but the spread inside the AI group was even wider.
You picked up a ticket in an async library you'd never touched. Forty minutes later it was merged. Tests green, reviewer happy.
A week later someone asks you in review why the cancellation behaves the way it does in that file, and you realize you can describe what the code does but not why it has to be that way. You could read it. You could not have written it.
That gap has now been measured, and the number is stranger than the usual argument about it.
What the randomized trial actually found
Anthropic ran a randomized controlled trial on this question. Fifty-two mostly junior engineers, all with at least a year of weekly Python experience, learned Trio — an asynchronous library none of them had used. Half worked with AI assistance, half coded manually. Everyone completed two coding tasks and then took a quiz covering debugging, code reading, and conceptual understanding.
The AI group finished roughly two minutes faster. That difference was not statistically significant.
The quiz results were. The AI group averaged 50 percent. The manual group averaged 67 percent. The single largest gap appeared in the debugging questions.
Seventeen points is not a rounding error. It's the difference between someone who can maintain a system and someone who can only add to it.
The finding underneath the headline
"AI makes developers worse" is the reading that travelled. It's not what the data says.
Inside the AI group, the spread was wider than the gap between the AI group and the manual group. Some AI users scored below 40 percent. Others scored 65 percent or higher — essentially level with the people who had coded everything by hand.
The low scorers shared a pattern: complete delegation of code generation, progressive reliance that deepened across the session, or using AI to solve bugs rather than to explain them.
The high scorers shared a different one: asking follow-up questions after the code appeared, requesting explanations alongside generation, or using AI only for conceptual questions while writing the code themselves.
The variable was never how much AI you used. It was whether you stayed in the loop while you used it.
The other half nobody quotes
Anthropic's earlier observational research found AI can cut task completion time by as much as 80 percent — on tasks where the developer already had the relevant skill.
Two findings that look contradictory and aren't. Up to 80 percent faster on the familiar. Seventeen points worse on the unfamiliar. Same tool, same person, potentially the same afternoon.
Call it your skill line. Everything you've built before sits inside it. Everything new sits outside. The line moves across a career — that movement is the career — and AI behaves in opposite ways on either side of it.
Why debugging takes the worst of it
Reading code is recognition. You see a construct, it looks familiar, it makes sense, you move on. Recognition feels like understanding. It's fast, it's cheap, and it is not the same skill as producing the thing from nothing.
Debugging needs what recognition can't supply: a causal model of the system. You hold a theory of why something behaves the way it does, predict what a change will do, get it wrong, and update. The being-wrong step isn't a side effect of learning. It's the mechanism.
When AI produces working code, you get the artifact without the search. The search was the part that turned into skill.
You can only debug a mental model you actually built.
What this does to a career
Engineering careers ran on cheap failure. Small, low-stakes work. You did it badly, someone corrected you in review, you did it slightly less badly. Multiply by three years and you have an engineer.
Small, well-specified, easily-verified work is exactly what AI absorbed first. What disappeared from the bottom of the ladder wasn't the jobs — people are still being hired into them. What disappeared were the mistakes. And the mistakes were the tuition.
Seen that way, several separate complaints turn out to be one complaint: you can read code you couldn't have written; reviewing AI output doesn't teach the way writing bad code did; throughput is up while your sense of progress is flat; senior engineers got disproportionately more valuable; the market rewards people already inside it.
That last pair follows straight from the mechanism. Senior engineers bought their skill line before the price changed, so nearly all their work now sits inside it and collects the multiplier. Someone three years in gets the multiplier on a smaller surface and pays the tax on a larger one.
The limits of the evidence
Two studies are not a law of nature.
The trial measured comprehension immediately after the tasks, not skill six months later. It also measured learning something new to you, not general programming ability — which is exactly why a blanket verdict on AI fails and the skill-line framing works.
The direction has been reproduced independently, though. A peer-reviewed study from the University of Maribor ran a ten-week experiment with 32 undergraduates learning React. LLM use for code generation and debugging correlated negatively with final grades. LLM use for explanations did not. Different population, language, and timescale — same split.
How to draw your own line
Everything here is compatible with using AI constantly. It's about using it differently on one category of task.
The discipline collapses into a single question asked before you prompt: have I built this kind of thing before?
If yes — delegate hard. That's where the 80 percent lives, and there is nothing noble about hand-typing a migration you've written forty times.
If no — the mode changes. Ask for the explanation before the code. Ask a follow-up after it arrives: not "does this work" but "why this and not the obvious alternative." When it breaks, take the first swing yourself before pasting the stack trace anywhere. Then explain the finished thing out loud to someone with nothing open in front of them; the holes surface in about a minute, roughly a week faster than code review finds them.
Keep a fence log — a running note of what moved from outside your line to inside it this month. If a month passes with nothing on it, that's the signal, and no performance review is currently designed to catch it.
Key Takeaways
In a randomized trial, developers learning an unfamiliar library scored 50% with AI assistance versus 67% coding manually — a 17-point comprehension gap.
The speed gain in that trial was about two minutes and was not statistically significant.
Debugging showed the largest gap of any question category.
Interaction style mattered more than AI use itself: delegating code generation scored below 40%, while asking for explanations and follow-ups scored 65% or higher.
Separate observational research found AI can cut task time by up to 80% on work where the developer already has the skill.
Practical rule: delegate freely inside your existing skill line; interrogate rather than delegate outside it.
FAQ
Does using AI make developers worse at coding?
Not by itself. The randomized trial found a 17-point comprehension gap when developers used AI to learn something unfamiliar, but the spread within the AI group was wider than that gap. Developers who asked for explanations and follow-up questions scored 65% or higher — level with those who coded manually. Developers who fully delegated code generation scored below 40%.
How much faster does AI actually make developers?
It depends entirely on whether the task is familiar. Observational research found up to an 80% reduction in completion time on tasks where the developer already had the relevant skill. In the trial on unfamiliar work, the time saving was around two minutes and was not statistically significant.
Why does AI assistance hurt debugging skill the most?
Debugging requires a causal model of the system — a theory of why something behaves as it does, tested by predicting, being wrong, and updating. Reading AI-generated code is recognition, which feels like understanding but doesn't build that model. In the trial, debugging questions showed the widest gap between the AI and manual groups.
What is the right way to use AI when learning a new library or framework?
Ask for conceptual explanations before generating code, ask follow-up questions after code appears, and attempt the first debugging pass yourself. These were the specific patterns associated with scores of 65% or higher in the trial. Avoid full delegation, gradually handing over more of the work, and using AI to solve bugs rather than explain them.
Are junior developers more affected by AI-related skill loss?
The trial studied mostly junior engineers, and the mechanism predicts a larger effect on them — not because of ability, but because more of their work sits outside their existing skill line. Senior engineers get the speed multiplier on a larger share of their tasks and pay the comprehension tax on a smaller one.
Should junior developers avoid AI coding assistants entirely?
No. Avoiding them removes a genuine productivity gain on familiar work and doesn't address the actual mechanism. The evidence points to changing the mode of use on unfamiliar work — explanation and interrogation instead of delegation — rather than reducing AI use overall.
Related reading
How to Let AI Think for You Without It Thinking for You — the same trade at the level of a single decision
AI Won't Take Your Coding Job. It Will Change It. — what the role itself becomes
AI Writes Code That Looks Right — why recognition fails you at review time
You're Using AI Like a Search Bar — the interaction-style point, applied broadly
The bottom line
Your employer measures throughput. AI raises throughput. There is no dashboard anywhere in your company for "hasn't learned anything new in seven months."
Your speed is visible to everyone. Your stagnation is visible to no one. That asymmetry is the whole problem.
The engineers who come out of this decade ahead won't be the ones who avoided AI, and they won't be the ones who used the most of it. They'll be the ones who noticed which side of the line they were standing on, and were willing to be slow on purpose about a small, specific slice of their work.
Adam Jaber is a software engineer who writes Simply Explained — complex topics, made simple. No jargon, no hype.
