# Learning With AI Coding Assistants: Why Careful Review Isn't Enough

**Short answer:** carefully reading AI-generated code is what learning researchers call the *read-only condition* — the control group that consistently loses. Retention is driven by prediction error: the gap between what you expected and what turned out to be true. If you never commit to an answer before the AI gives you one, there's no gap to learn from, no matter how attentively you read. The fix is to spend about sixty seconds writing down your own answer first.

That's the whole argument. The rest of this post is the evidence, the mechanism, and a concrete habit you can start on your next ticket.

![](https://cdn.hashnode.com/uploads/covers/6a44b5d24b41ab0145e5cf63/f0422efa-51af-4f42-95af-1df1347e8ddc.png align="center")

* * *

## Key takeaways

*   **"Use AI, but review the output carefully" describes the losing arm of the experiment.** Attentive study of a correct answer is the standard control condition in pretesting research.
    
*   **Struggle is not the active ingredient. The guess is.** Learning is driven by a violated prediction, not by time spent suffering.
    
*   **Your confidence signal is corrupted.** Looking something up inflates self-assessed knowledge even when the lookup fails, so "I understood that" is not evidence that you did.
    
*   **The habit is Predict → Prompt → Diff → Defend**, and it costs roughly a minute on problems that are worth it.
    
*   **Apply it selectively.** Concurrency, query plans, auth, money. Skip it for boilerplate, config, and throwaway scripts.
    
*   **Kept over time, your predictions become a calibration log** — the only practical instrument for measuring your own engineering judgment.
    

* * *

## What does the research actually say?

In 2025, Mahir Akgun and Sacip Toker published [*Struggle First, Prompt Later*](https://arxiv.org/abs/2504.10249), a set of three experiments on undergraduates in an intermediate statistics course.

In Study 1, every participant faced a cybersecurity problem requiring a two-way chi-square analysis they had not been taught. Every participant got ChatGPT. The only manipulated variable was **when**:

| Group | Procedure | Baseline recall | Final transfer test |
| --- | --- | --- | --- |
| No-pretest (n=31) | ChatGPT available immediately | 72.9 | **78.6** |
| Pretest (n=29) | Had to attempt the problem first | 71.0 | **91.9** |

The baseline scores were statistically indistinguishable, so the groups started level. The difference on the transfer test was significant with a large effect size (Cohen's *d* = 1.54).

Study 3 added a wrinkle worth knowing: the pretest advantage was **larger on open-ended, decision-heavy tasks** than on formulaic ones. Which maps unpleasantly well onto the work that separates a senior engineer from a junior one.

### The honest caveats

*   Sixty-odd statistics students, one session, one institution.
    
*   Chi-square analysis, not a production codebase.
    
*   No long-term retention measure.
    

Nobody has run the equivalent study on engineers over two years. Anyone claiming otherwise is overstating the literature.

But this study isn't load-bearing on its own. It's one instance of the **pretesting effect**, replicated for decades across text, video, vocabulary, and conceptual material: attempting to answer before instruction beats studying the correct answer for the same duration — *even when the attempts are wrong*.

* * *

## Why doesn't careful review work?

Because care changes attention, not architecture.

Learning is driven by **prediction error** — the discrepancy between an expected outcome and an actual one. A prediction, then a contradiction, then an update. It's why a bug you chased for four hours is permanently yours and a bug explained to you in standup is gone by Friday.

If no prediction was ever formed, there is no discrepancy. The correct answer arrives, it parses cleanly, you close the tab.

This is the failure mode of reviewing AI output attentively. You can read generated code with real rigour and still be running the control condition, because attention applied to a correct answer produces **recognition**, not knowledge.

> You cannot learn from a correction you never earned.

Recognition is indistinguishable from understanding while you're inside it. Clean code that makes sense line by line reports back as *I know this*. And you do — the way you know a song you can't sing.

**Note what this reframe rules out.** The popular version of this argument says juniors need more struggle. That's wrong, or at least badly targeted. Struggle is a proxy. What you need is the guess. You can keep almost all of AI's speed and still get the learning, because the expensive part was never the grinding — it was the sixty seconds of commitment that the grinding happened to include.

* * *

## Why do developers resist this?

Because the method that works feels worse than the method that doesn't.

In [*Searching for Explanations*](https://doi.org/10.1037/xge0000070) (Fisher, Goddu & Keil, *Journal of Experimental Psychology: General*, 2015), nine experiments showed that searching the internet for an explanation inflates people's estimates of their own internal knowledge — they rated themselves better able to explain *unrelated* topics afterwards.

The critical detail: **the effect appeared even after unsuccessful searches.** Lookups that returned nothing useful still raised confidence.

Map that onto your terminal. The instant a fluent, well-structured, correct-looking answer renders on your screen, your self-assessed understanding rises — whether or not anything in your head moved. The signal you would normally use to detect that you didn't learn something is precisely the signal that gets corrupted.

There's a companion result from memory research whose title says everything: making related errors facilitates learning, *but learners do not know it*.

> AI didn't just automate the work. It automated the feeling of having done the work.

So a method with better outcomes and worse ergonomics competes against a method with worse outcomes that feels like mastery. Nobody has to be lazy for the wrong one to win.

* * *

## The 60-second pretest

![](https://cdn.hashnode.com/uploads/covers/6a44b5d24b41ab0145e5cf63/6164730f-5f2a-4eeb-b6d4-a32d7eb9e778.png align="center")

Four steps. Move the thinking from after the answer to before it.

1.  **Predict** — write what you think the answer is. In the file, as a comment. An unwritten thought silently reshapes itself to match whatever the model says.
    
2.  **Prompt** — ask the real question. Do not soften it into something that will agree with you. You want the answer, not the ratification.
    
3.  **Diff** — locate the exact line where your answer and its answer diverge. "Mostly right" is not a diff.
    
4.  **Defend** — explain the divergence out loud with the screen turned away. If you can't, you read it. You didn't learn it.
    

### What a real prediction looks like

"Probably a database thing" is not a prediction. It's a mood. A prediction must be capable of being wrong in a way you'd notice.

An endpoint returning orders with customers is slow in production and fine locally. Before prompting:

```python
# PREDICTION — before asking
# Cause:      N+1. One query for orders, then one per order for .customer
# Fix:        select_related("customer")
# Expected:   1 query instead of 1 + len(orders)
# Confidence: 7/10
# Unsure:     whether the serializer also touches customer.address,
#             which would need prefetch_related on top
```

Six lines, roughly forty seconds. Then prompt, and exactly one of three things happens.

**You were right.** The confirmation lands on a position you committed to — a different memory event from reading a fact cold.

**You were wrong.** Better. You now hold a labelled gap: *I thought the serializer was the problem; it was the pagination count query.* Lessons with handles are the ones you retrieve at 3 a.m. eighteen months later.

**The model was wrong, and you caught it** — because you arrived holding a model instead of an empty page.

That third case is chronically underrated. **You cannot audit an answer you have no independent view of.** Every "review the output carefully" workflow assumes a reviewer with a position, and a workflow of pure review never produces one.

### A second example: concurrency

Prediction discipline pays most where the failure is invisible in dev. Two requests decrement the same inventory row:

```python
# PREDICTION — before asking
# Behaviour: read-modify-write. Both txns read stock=1, both write stock=0.
#            One sale is silently lost. Wrapping in a transaction does NOT fix it
#            at READ COMMITTED — it's an atomicity problem, not an isolation one.
# Fix:       SELECT ... FOR UPDATE, or an atomic decrement with a stock > 0 guard
# Unsure:    whether our default isolation level is READ COMMITTED or REPEATABLE READ
```

Write that, then ask. The `Unsure:` line is the highest-value line in the block — it tells you where to look when the answer arrives, instead of skimming the whole response uniformly and absorbing none of it.

* * *

## When should you skip it?

Applying this to every prompt is a tax on your throughput, and you'd abandon it inside a week.

![](https://cdn.hashnode.com/uploads/covers/6a44b5d24b41ab0145e5cf63/3f963f7f-fc33-4464-bdbb-f473e1d07cbd.png align="center")

**Pretest it when:**

*   The code touches money, authentication, or user data
    
*   Concurrency, locking, or transaction boundaries are involved
    
*   You're reasoning about query plans or behaviour under load
    
*   You'd be asked to explain the failure in a postmortem
    
*   You've hit this pattern three times and still have no mental model of it
    

**Just prompt it when:**

*   It's boilerplate or config you will never reason about again
    
*   It's syntax in a language you're passing through
    
*   It's a one-off script with no second reader
    
*   It's regex, a cron string, or CSS you can verify by eye
    
*   Being wrong costs a rerun
    

The sorting rule: **pretest anything you'd be embarrassed not to understand when it breaks at 3 a.m.** There is no judgment in a cron string. Let the machine have it.

* * *

## What this builds over time

Keep the prediction comments — a scratch file, a note, anywhere — and you accumulate something almost no engineer has: a record of what you believed before you found out, how often you were wrong, and **in which direction**.

Patterns surface fast. You systematically underestimate how much latency is I/O. You assume the framework handles the transaction boundary. You reach for caching before you've read a query plan.

That's calibration, and it's the closest thing to an instrument for measuring engineering judgment. Every "seniority is judgment now" post asserts that judgment matters. Very few name a mechanism that produces it.

The mechanism is unglamorous: write down a wrong answer before you get the right one, repeatedly, for a couple of years.

* * *

## The one-line version

The gap between the engineer who compounds and the engineer who merely ships is not tool choice, hours, or discipline. Both are careful. Both review the output.

The gap is sixty seconds — whether you committed to an answer before you saw one.

Take one problem tomorrow. Write four lines of what you think is happening, then ask. See where the diff lands. You'll probably be wrong; that was always the point.

* * *

## FAQ

### Does using AI coding assistants make you a worse developer?

Not inherently — the mode of use decides it. Delegating code generation and reading the result produces recognition without retention. Using the same tool for conceptual inquiry, or after committing to your own answer, preserves and often accelerates learning. The tool is neutral; the sequence isn't.

### Is carefully reviewing AI-generated code enough to keep learning?

No. Careful review increases attention but doesn't create a prediction to violate, and prediction error is what drives retention. In learning research, attentive study of correct material is the standard control condition — the one that loses to guessing first.

### What is the pretesting effect?

The pretesting effect (also called the prequestion or errorful generation effect) is the finding that attempting to answer questions *before* studying material improves later retention compared with studying for the same amount of time — even when the attempts are wrong. Incorrect guesses appear to prime attention and improve how corrective feedback is encoded.

### How long does a pretest actually take?

About 30–60 seconds for a four-line comment naming the cause, the fix, your confidence, and what you're unsure about. On a task where you'd spend twenty minutes anyway, that's a rounding error against the retention difference.

### Should juniors avoid AI coding assistants entirely?

No, and abstinence advice tends to be ignored anyway. A junior who predicts before prompting gets AI's speed *and* the feedback loop that builds judgment. A junior who prompts first gets the speed and a flat skill curve — which is the actual risk, not the tool itself.

### How do I know if I'm actually learning from AI or just recognizing the answer?

Close the tab and explain the solution out loud, including why the obvious alternative is wrong. Recognition collapses the moment the screen goes away. If you can only follow the reasoning while looking at it, you read it — you didn't learn it.

* * *

## Related reading

*   [Why Your App Is Fast in Dev and Dying in Production](REPLACE_URL) — the N+1 example above, in full
    
*   [Your Database Transaction Didn't Save You From This Bug](REPLACE_URL) — the concurrency example, in full
    
*   [How AI Turns Words Into Meaning — and Where It Quietly Fails](REPLACE_URL) — why generated answers arrive sounding confident regardless of correctness
    
*   [AI Won't Take Your Coding Job. It Will Change It.](REPLACE_URL) — what the job is becoming
    
*   [Start Here](REPLACE_URL) — the full Simply Explained map
    

**Sources:** Akgun & Toker, [*Struggle First, Prompt Later*](https://arxiv.org/abs/2504.10249) (2025) · Fisher, Goddu & Keil, [*Searching for Explanations*](https://doi.org/10.1037/xge0000070), *JEP: General* 144(3) · Pan & Carpenter, *Prequestioning and Pretesting Effects: A Review*, *Educational Psychology Review* 35(4) · Sparrow, Liu & Wegner, *Google Effects on Memory*, *Science* 333(6043)

* * *

*What's the last thing you shipped that you couldn't have explained a week later? I'd like to know whether this pattern matches what you've seen — or where it breaks down.*