Rovo Dev CLI Ralph Wiggums Loop for Large Scale Test Refactoring

What is the “Ralph Wiggums” approach?

A lightweight AI loop that repeatedly points an agent at a small, explicit spec and asks it to complete just the next step, record learnings, and stop. It’s fast to iterate, easy to constrain, and great for well-bounded refactors where success is objectively testable.

Why this approach was a good fit for my use case

I needed to refactor a large frontend codebase replacing up to 2700 occurrences of a heavyweight test wrapper with minimal specific providers, improving setup speed while preserving test coverage.

While every case had to be individually considered, executing a Rovo Dev loop was a good fit because:

• Success is easy to validate: if the tests are green, we’re good with that file.
• Failure is an option: more complex cases could be flagged and deferred.
• Scope is tightly bounded: each iteration focuses on a single test file.
• Applicable lessons learned: many tests follow similar patterns, so Rovo can apply a similar recipe while adjusting based on prior conclusions.

The workflow I used: a Rovo Dev Ralph loop tuned for large scale changes

First, identify target files

This AI loop requires a list of tasks to track progress. And I knew some test cases would be easier replacements providing higher performance boosts.

I prompted Rovo Dev to scan the codebase, and build the Spec for me, listing the top candidates worth refactoring.

In retrospective, a similar Rovo Dev loop could have been used to create an exhaustive list scanning a package at a time, and breaking the results into separate Specs based on change complexity and impact.

Refine it into an actionable Spec

Once I had the list of target files, I needed to provide some basic principles for the refactoring. For this, I linked to:

• The Confluence page where we had discussed the refactoring approach.
• And a couple of previous PRs that addressed some of these cases manually.

Rovo Dev CLI has access to the contents behind these links, but I asked Rovo to write down instructions in the Spec focusing on the main principles and summarizing relevant context, for easier access and higher performance.

One iteration = one file

Changes for a test refactor are compartmentalized by their own nature. By directing Rovo Dev to only modify a single file per iteration I could keep the scope of each pass tight.

Further iterations just needed to find the line after the last file path that had already been optimized.

Bottleneck: local test execution

The most limiting factor was the speed of my own computer when running tests locally to validate the outcome of each change.

While this wasn’t a problem when running this overnight, it might make more sense to attempt changes on multiple files iteratively, then run the tests as a whole batch once – so test engine startup happens once, and parallelization and caching can kick in. Further loop cycles could then start with a fix attempt, or a revert, where appropriate.

Failure proof

Some of the replacements were far from trivial, and some tests could flake out during high load. Because of this, I modified the loop to provide more nuanced completion markers for each processed file:

• flag timeouts, since flakiness in the local dev loop shouldn’t necessarily result in a rollback
• identify files where only a few tests had been optimized, as those may need further passes or manual attention

But in practice, most files came out with perfectly working code.

For large scale changes lessons learned need to be summarized

The regular loop approach would write what was processed, what went well, and gotchas detected. Per each iteration.

This doesn’t work when you’re processing hundreds of files and you’re dumping conclusions into a single Spec. Further iterations of this same logic would cause the context of this file to blow up beyond what the AI agent could consider.

As a mitigating factor, I updated the prompt to summarize lessons learned and consolidate with existing lessons instead of adding a more verbose approach for each completion.

APPEND learnings only in the SPEC’s **Learnings (for subsequent runs)** section, in compact form (one line per entry, no duplication).

This change kept the file size in check, ensured that most lessons learned were still considered, and made further iterations more effective than the first ones.

The outcome

• 160 files of changes
• Almost everything working out-of-the-box (since I asked timeouts to not be rolled back)
• Close to a thousand tests being many seconds faster than they used to be

All done by Rovo Dev CLI overnight.

Totally worth it.

Will do it again.