At the Atlassian Summit in June this year we announced Bonfire, a tool for rapid bug reporting and session-based testing. In the five months since then we’ve sold over two thousand Bonfire licences, but had it not been for the generous support of my colleagues and my now three year old son, Bonfire wouldn’t exist. My name is Andrew Prentice, I manage the QA team at Atlassian and this is the story of how Bonfire came to be.
The idea for Bonfire arose from a time tracking study we did in the QA team back in 2009. We looked at four hundred hours of testing and learnt that 16% of our time was spent reporting bugs in JIRA. Whilst several factors contributed to this, the biggest culprit was the effort required to create comprehensive bug reports. Often that required multiple round trips between JIRA and the software being tested in order to collect and complete the issue meta data, attaching screenshots can be convoluted if you need to annotate them and the context switching involved disrupts testing productivity. It meant a lot of time was spent not testing. If we could halve the time spent creating issues then every QA engineer would gain back a month a year for more testing – the same as hiring an additional person for every twelve already on the team.
The initial requirements were straight forward. We needed a way to:
- Create issues without leaving the context of the application being tested.
- Take screenshots and annotate them without switching apps.
- Populate issue meta-data automatically.
Like many companies, Atlassian has a set of values, but unlike any other company I’ve worked for these values really mean something. We question ourselves and each other continually about whether what we’re doing upholds our values and those answers are decisive.
To begin with it was definitely progression via trial and error as I worked on building the screenshot drawing tools. To complicate matters further trigonometry wasn’t my best class at school so I found implementing a tool to draw circles tough. There were many nights of hair-pulling, inexplicable failures and circles rendering in unexpected places all over my screen. Man I hated circles. However, with time and Wikipedia’s trigonometry entries, my understanding of SOH-CAH-TOA grew enough to make it work.
By June 2010 I’d written the screenshot and annotation functionality and integrated it with Pete’s Firefox extension spike. At the Atlassian Summit that year, I showed it to anyone who had sought me out to talk about testing. I wanted validation that this hadn’t turned into a vanity project and that the idea still had merit, but I wasn’t expecting people to ask when it would be available commercially. I came back even more determined to finish the project, but also thinking about ways to make it more useful for any tester, not just Atlassians.
I released it internally in October 2010. Here’s how it looked:
It’s internal name was Excalibur. I had come to the conclusion that this could become more than a tool for rapid issue reporting. It had the potential to become a swiss army knife for web testing and by all accounts Excalibur was a pretty good knife. It takes more time to explain that name than was put into choosing it (unlike its final name Bonfire – more on that later), but the name served as a reminder to build a tool that testers could wield in their own way, rather than a tool that forced testers to behave in specific ways.
We’ve found that traditional testing approaches aren’t suitable for the rapid pace of Agile development, so the only manual testing we perform is exploratory testing, which traditional test tools don’t support. Customers often ask us to build test case management into JIRA, but we don’t want to build something that we wouldn’t use ourselves. We’ve trialled James and Jon Bach’s Session Based Test Management (SBTM) method and although we liked it a lot, it was cumbersome to manage it with JIRA and the few SBTM tools around weren’t mature enough for our needs.
Adding support for session based testing in JIRA, therefore was at the top of my list for extending Excalibur. What’s more, the idea of providing an atypical test management solution specifically targeted at Agile teams caught the attention of JIRA’s product management. In Part II, I’ll explain how their support plus some wheeling and dealing transformed Excalibur from an internal tool into Bonfire the commercial product.