Strategies to review AI code

As AI becomes faster and smarter at an exponential rate, one thing becomes the bottleneck: you 🙌

Well, if nothing else, your capacity to review giant changes made by AI stays mostly the same 💁🏼‍♂️

You cannot keep up with thousands of lines changes from an AI loop maxxing run just by reading line by line.

Here are a few strategies to consider instead 👇

Giant changes, giant review

The new trends seems to be looping AI on its own to implement full products all at once.

This makes sense in theory, but leaves a giant hole on the other side: review 🤔

I see 3 options:

I trust 100% AI code, no review, just ship
I just skim the code, do a few manual testing, and ship
I review the code for real

Now, for the current generation of AI (and for a product you care being reliable), we may agree option 1 is out 🙂‍↔️

What about the other two? 👀

Vibe coding

Option 2 is possibly the definition of vibe coding: ask, test manually, ship.

This works, short term 😬

As time passes, two harsh realities become apparent:

You have no idea how the code works
Bugs start to appear (fast)

These two add up:

Keep prompting and add more and more bugs as you fix previous ones, because you don't understand the code enough to fix them manually if needed 🙌

Not ideal.

The bottleneck is always the same: you cannot keep up with AI in a loop 💁🏼‍♂️

Slow AI

Option 3, I call it Slow AI ™️

AI reads and writes fast. As such, it can load the context of the code and any change needed faster, and implement it even faster than that.

Here, I would argue, the match is lost. Let AI write 🙌

Writing is not your responsibility anymore. But reviewing is.

As such, you should slow down AI to follow your pace. And there are a few smart ways to do it.

Number 1: Plan a lot before the implementation, and keep the scope small

Ask the AI to break down the feature in testable intermediate steps. The AI may be able to one-shot all in one prompt, but you don't, so break it down.

Then read the plan, in the details. Make sure all the assumptions and goals are correct before letting the AI go.

Number 2: Let the AI go. Completely, until it's done. No intermediate check.

The plan should carry the AI all the way to the end. It will write, check, review its own diff, rewrite.

You stay out of it, and wait until the thumbs up for review.

Number 3: Skim the code. Look for weirdness only.

That's right. You may not need to do a full line-by-line review. Two situations:

You already understand the code, and the changes are only obvious additions
This is only a small step in a bigger feature

With (1), nothing really new has been added, so you don't lose any knowledge about the codebase.

(2) is interesting.

Small reviews, big check at the end

Remember when we broke down the plan into smaller steps?

Well, this serves you to review smaller set of changes. But since the features is not complete, you don't need a full detailed review.

But skimming still makes you lose some context, which will compound over time 🤔

Your skimming goal is to fix glaring issues only. And you retain a little more understanding, even if not complete.

At the end, you have a scattered view of the code, but you are not lost 👀

Now the "real" review begins:

Use AI to really understand the changes, until you understand what has been added as a whole 🪄

Do a full (slow) audit of the code. Any piece you don't understand, ask AI. Every broken or weird implementation, ask AI to review, explain, and fix if necessary.

Repeat "yourself" in a loop, until everything looks correct.