This guest blog post is part of an Atlassian blog series raising awareness about testing innovation within the QA community. You can find the other posts in this series under the QA Innovation tag.
The post is written by Catherine Powell, a principal at Abakas, a software consulting company. At Abakas, she provides engineering management, development, testing, and process consulting services.
Ahh, the glories of software creation. We have an idea, and an idea turns into a list, and a list turns into code plus more lists, which turns into more code and new machines in production ready to share our great new software with the world. And there are a few caffeine-fueled late nights, and there are a few company-sponsored lunches and demos. And then we’re ready… just as soon as we run one last regression test cycle. And it’s going to take way too long. We’re all excited and ready to go, and we have to stop to wait for test. It’s the last thing to finish, and there are never enough testers during those end crunch times, and we keep having to cut corners to make releases happen on time. Plus, it frequently points out that we made the same mistakes we always make. So then we have to patch things and fix configurations and tweak it. That darn testing!
So how do we fix it?
How do we keep testing from being a bottleneck?
Simple. We do less of it. We minimize that end-of-cycle testing holdup. And we do THAT by building trust in our development and in our software creation, deployment, and maintenance process. Test cycles, particularly regression test cycles, are ultimately driven by fear. We do a “test pass at the end” because we do not trust that the activities that lead up to this point have resulted in robust, deployable software that does what we meant for it to do. Fix that lack of trust, and we can skip the extended test cycle.
There’s an old saying about starting off on the right foot. We don’t always have the luxury of starting (or restarting), but we can find the right foot, even on an ongoing project. To improve our testing, to make it more efficient, we have to address the whole software development life cycle. We must find our underlying fear and fix it instead of hiding behind an ever-larger test cycle.
The tricky thing about the fear driving the “test pass at the end” is that it’s frequently justified. We keep finding bugs, or configuration errors, or data problems, or deployment failures. We can’t stop testing when it keeps finding things that we really do need to know! Step one to reducing this test phase is finding the true underlying problems causing our fear, and fixing those. In other words, to make testing more efficient, we have to fix everything else.
The Underlying Problems
Let’s look at some of the more common problems that cause fear, and what we can do about it:
- We fear that our deployment didn’t work or wasn’t complete, so we test. Instead, we can automate deployment so code gets into QA the same way it gets into production. That way we only have to test in one environment (QA), not two.
- We fear that a developer forgot to check something in or didn’t make the build properly, so we test. Instead, we can set up a build system and only test – or deploy – from there. Again, it means we only test a build once, not once per environment.
- We fear that a new feature accidentally broke the old way things worked, and that customers want the old way and the new way to both work, so we test. Instead, we can create a regression test suite and automate it, so the build system can run it every time we check in code. There’s no need to wait until the end to find out that something regressed.
- We fear that some underlying change – a browser version or OS patch – broke our software, so we test. Instead, we can schedule that testing at a different point in the cycle. Tie it to the release of the software we’re worried about, not to our own release schedule. Let our regression suite (see above) help us find major problems, watch the release patterns of the software with which we interact, and don’t test just because something might have changed. Reserve this for when something actually did change.
- We fear that several features coming together at the very end will break each other, so we test. This one we still need to test! We can – and should – try to bring in features serially over time, but if the development team is larger than about two people, there’s going to be a bit of a rush trying to get things in before some deadline, whether that deadline is a sprint, a release, or just Friday. Minimize it where possible, and then move everything else out of the way so only this much smaller task is left.
Test smarter, not harder
Fear causes us to take on large test cycles near the end of a release (or sprint or story or whatever the “chunks” are in a given project). We perform slow, repetitive tasks with a relatively low value, just because of that fear. Eliminate the source of the fear, automate the repetitive and low value tasks, and make that end testing cycle less of a bottleneck. Get the software on the right foot, and you’ll free testers to do the activities that not only assuage fear, but that truly provide value.
Catherine Powell has worked with a broad range of software, including an enterprise storage system, a web-based healthcare system, data synchronization applications on mobile devices, and webapps of various flavors. She is an author, speaker and a mentor to engineers and technical managers. Catherine focuses primarily on the realities of shipping software in small and mid-size companies. Catherine’s published works, blog, and contact information can be found at www.abakas.com.