When you use AI to write code, check whether you can explain the parts you will be responsible for maintaining. A change can pass its tests without giving you enough practice to diagnose a different failure later.
This distinction showed up in a January 2026 study from Anthropic. Researchers asked developers to complete tasks using an unfamiliar Python library, then tested their understanding. The group with AI assistance averaged 50% on the quiz. The group without it averaged 67%, a difference of 17 percentage points. The small improvement in task speed with AI was not statistically significant.
The study does not tell us that every developer learns less with AI. It gives us a specific reason to check understanding separately from task completion.
What the study measured
Participants worked with Trio, a Python library for managing tasks that can run concurrently. Concurrency lets a program make progress on more than one task during the same period, such as waiting for several network requests.
After the coding work, participants answered questions about concepts they had used, code behavior, and debugging. Researchers found the largest performance gap in debugging questions. That matters because maintaining code often requires identifying why something failed, not only recognizing what a successful example looks like.
This was a short experiment involving an unfamiliar library and a quiz soon after the task. It did not measure years of professional development or prove that AI use causes lasting skill loss.
A practical example during review
Suppose an AI assistant adds retry logic to a background worker. The tests pass, and the code looks reasonable.
Before approving it, consider a case where the server completes the request but the worker never receives the response. What will happen when the worker tries again?
For a request that fetches information, repeating it may be acceptable. For a request that creates an order, the consequences could be different.
The example is hypothetical, but it gives the review a concrete direction. Look for how the API handles duplicate requests. Then check whether the worker follows that behavior and whether a test covers the uncertain response.
You do not have to memorize every line. You should be able to trace the behavior that matters to the change.
How to use AI while investigating
Anthropic’s researchers observed different patterns within the AI group. Participants who asked conceptual questions or sought explanations sometimes showed stronger understanding than those who delegated most of the work. Those patterns are useful observations, but they were not separate randomized tests proving that one prompting technique causes better learning.
A reasonable practice is to use the assistant to help investigate a question you have already identified.
Instead of asking it to explain the entire pull request, try a narrower request.
Show where this worker decides to retry. Explain what it knows about the first request at that point. Identify any behavior that depends on the external API.
Read the cited code yourself. If the explanation depends on an API guarantee, check the documentation. The assistant’s response gives you a path to inspect, not a reason to skip inspection.
Check understanding with a different case
After reviewing the original change, change one condition in the example.
What if the worker restarts between attempts? What if two workers pick up the same job? Which test would reveal the problem?
These questions are editorial suggestions, not a training method validated by the Anthropic study. They are a practical way to make a review more specific.
If you cannot answer one of them, you have found something worth investigating. Record the answer where the next maintainer will find it, such as a test, a code comment explaining a constraint, or a note in the pull request.