Like most software engineers nowadays, I have AI coding agents handling the bulk of my work. Where I possibly differ with my peers (based on my personal experience working with other developers) is that I actually spend the time reading the code that was generated by the agent. I don’t go through the changes line by line with a fine-toothed comb, but I do read through everything before merging it in. There’s an ongoing argument whether we should spend time reviewing AI-generated code, but that’s a topic for a different day.

What I do want to discuss is how we do code reviews that spawn from an AI service. My tactic previously was that when the coding agent finished its work, I’d open the files containing the code changes that affected the behavior of the application and defer everything else to later. In particular, I wouldn’t read the generated tests until after I finished going through the implementation changes. The developers who do take the time to read code changes without deferring it to another AI service typically go through the same process.

However, I’ve come to believe that this is the wrong way to review AI-generated code. Instead of checking the code changes, we should first take a look at the automated tests that the agent included as part of its work.

Why Read the Tests First?

Engineering teams primarily use automated tests to ensure that the application behaves as expected. But an equally important reason for their existence is to document what someone believed the code should do. Anyone who’s trying to understand how the application works under the hood can go through the application’s automated test suite. In a well-written test suite, you’ll have clear descriptions of behavior, examples of expected input and output, and how failures are handled. It should give developers a solid grasp of the system’s internals to let them do their job.

In the days when we had to write code ourselves, developers were mainly responsible for having automated testing as part of their code changes. When we included tests alongside our code to implement a new feature or fix a bug, it required us to fully understand what we were working on. If we didn’t fully comprehend what we were building, the tests could help point us in the right direction. That’s one of the reasons why test-driven development was a popular development methodology, because it requires you to first articulate what you want before building it.

Now that most automated tests these days are produced by AI, we’re passing on the step of understanding what to build over to the coding agent. Sure, we’re the ones writing the prompts to fire off the work. But no matter how well-crafted your prompts are and how much better AI models are becoming at coding work, there’s a good chance the agent will go off on a tangent. It will implement something that isn’t quite what you asked for.

The implementation of a prompted feature or bug fix can be logically sound, structured perfectly according to project guidelines, and keep the regression test suite green. These days, coding agents are pretty good at producing code that looks and works well. But if they implement the wrong thing, then it doesn’t matter how great the code is.

To make sure that your intentions were followed by an AI coding agent, the real proof you have from the agent’s output is the automated tests that it wrote. That’s why I’ve begun to read the tests first in what I’m calling test-driven review.

What to Look for When Doing Test-Driven Review

For most modern programming languages and frameworks, AI does a thorough job of including enough test coverage. You have to be careful since they can go overboard with the number of test cases they create, along with including highly inefficient and unnecessary tests. But for the most part, the test coverage is enough to give us an idea of how well they did their job.

When digging through the tests as part of the test-driven review of a coding agent’s output, here are the top three areas I look for to ensure that my intent was understood and implemented well enough by the agent. It’s not a deep dive into everything I verify, but these spots give me a decent view of the actual implementation elsewhere in the changeset.

Test names that describe what the code does

The name given to an automated test case can reveal a lot about how well-written the underlying code is. A good clue for knowing at a glance if an AI-generated test is useful is by reading its name and determining if it explains the implementation or the intent. Let’s use the following two tests written for a Ruby on Rails application that processes payments through a gateway:

 test "returns a hash with a :success key" do
   # Test code
 end
 
 test "charges the specified amount when a user pays" do
   # Test code
 end

Without showing the test code, we can assume that the test suite is verifying the use of the application’s payment gateway. However, there’s a distinction between both scenarios. The first one looks to be verifying implementation, while the second one seems to verify intent. The first test scenario really only tells me what the code does, and the second tells me what behavior happens. The latter is essential for verifying any application. Code correctness matters, but a test that only demonstrates the code returns the right output tells you nothing about whether it does the right thing.

Missing test cases

I’ve found that, on its own, AI tends to follow mostly the “happy path.” Unless explicitly told, it rarely takes a look at areas of potential failure. When reading the code implementation on its own, it’s easy to miss these spots. The lack of automated tests is another clue that there’s a potential for something to blow up in the new code. Using another example, let’s say a coding agent created a new method called split_name and included the following tests:

describe "#split_name" do
  test "separates first and last name" do
    # Test code
  end
  
  test "handles middle names" do
    # Test code
  end
end

I can immediately spot some crucial scenarios that are not covered by these tests. What happens when an empty string is passed to the method, or a single string with no spaces? What about if someone enters four names or uses names with Arabic or Japanese characters? The split_name method might cover those scenarios, but I’m willing to bet that the coding agent forgot to address those conditions. If I notice a lack of test coverage in a given function, I know I need to pay close attention to the changes made by the AI.

Tests written to pass, not to catch

One of AI’s bad habits is to attempt to fix everything however it can. We can see it when a coding agent spends an inordinate amount of time going in circles when trying to implement something that isn’t working. Without explicit instructions, it often tries to cheat its way through making a check pass instead of addressing the underlying problem. A good example is an application that uses linting to ensure it follows the project’s coding standards. I’ve found that unless I have a rule in place, a coding agent will simply update the linter’s configuration to ignore a formatting error instead of fixing it.

Coding agents also love to do this with automated tests via mocking. Here’s another example for a Rails application using Minitest that integrates with a third-party CRM to keep customer data in sync:

test "does not sync suspended accounts to the CRM" do
  client = Minitest::Mock.new
  client.expect(:sync_customer, false, [Account])
  
  CustomerSync.new(client).call(@suspended_account)
  
  assert_mock(client)
end

Without diving too deep into Minitest, this test creates a new mock for a client object, and when that object receives the sync_customer method with an account, it will automatically return false without hitting the underlying implementation. While mock usage is a standard practice with third-party services, this test only confirms that the expected calls happened and doesn’t tell me anything about the code’s behavior. The name says suspended accounts don’t get synced, but the mock setup requires sync_customer to be called and assert_mock fails if it wasn’t. The test only passes when the suspended account was synced, the opposite of what it claims to check. Typically, these kinds of assertions are a sign that the coding agent wrote it only to ensure the test suite passed, not because it’s testing something important.

Test-Driven Reviews Are a Faster and More Reliable Way to Validate AI’s Output

Doing test-driven development in the past worked because it forced you to know what you wanted before spending time coding. You still need to know what you want, but coding agents do the articulating for you now, and they show their interpretation of your instructions through the tests they generate. Reading them first is how you can uncover what the agent thought you asked for and gives you the quickest indicator of whether the implementation is close to what you wanted or not.

Test-driven reviews don’t mean you should skip reading the code implementation. What it does change is how you read the result of an AI coding agent’s work. If you dig through the tests first, you’ll already know which parts of the implementation need more scrutiny, and you’ll have a short list of places that need your focus. It’s not the best use of your time to give every single change the same attention. Instead, by going through the spots that you’ve flagged by scanning the automated tests, your reviews will get faster, and you’ll catch more problems that matter.