Quality Health Monitor: Gauging your team’s quality maturity
Uncover your team’s quality concerns and assess your team’s maturity in 1 hour
Since 2010, Atlassian Quality Engineers (QE) have practiced the Quality Assistance model. We have definitely seen the benefits of this approach, especially when a QE is embedded in a team. Atlassian is an agile company, and our QEs ensure QA related activities and standards are optimized for their teams. Because different teams have different expectations, the meaning of "optimum state of quality" can differ. This raised several questions from QEs:
- What does this “optimum state of quality” look alike?
- What should be our journey towards it?
- Are there stages leading to that highest state? How can we show progress?
- Do we have the right assets and skills to get through each stage?
- What should we be prioritizing?
These questions can all be answered, but responses will vary depending on the team they support. It was a challenge to answer those questions at a department or organization level. It was also impossible to sum all the teams' responses to paint a picture for an organization’s state of quality because teams had different criteria. It became obvious that QEs should first define and agree upon the criteria. We call these quality attributes that teams should strive for.
There are many attributes out there. These attributes differ from one organization to another and can change over time.
There’s a lot of quality attributes to choose from!
Ideal solution for rating
We then looked for the best method to assess how a team lives up to these quality attributes. Since we were pushing for making quality everybody’s responsibility, we wanted something the entire project team can participate in. We don’t want another rating system that only had a single-view perspective. What we realized is we can re-use and re-purpose a popular Atlassian team workshop - the Atlassian health monitor!
A health monitor is an hour-long workshop that our teams at Atlassian have to assess performance as a team against 8 attributes. It's a good exercise to openly discuss and consider aspects teams probably have never thought before. It's a safe place to discuss people's different perspectives and pushes the team to address their concerns. The team can track their improvement over time through regular runs of this workshop.
So we produced the Quality Health Monitor (or QHM), which focuses on 8 targeted quality attributes that project teams rate themselves against. Have a look at our example - Quality Health Monitor sample.
How it works
Health monitors are easy to use and you can get started right away. Watch this video for more details. You can also watch this brief video - https://vimeo.com/227967631.
Additional tips when performing the QHM:
- Find a facilitator (external to the team) - All team members should participate, including the team’s QE.
- No more than 8 - This is the magic number for health monitors. If you go beyond that, then the team will be pressed for time.
- Remind the team of the objectives at the start of the session - The end goal is to drive the team’s improvement towards producing consistent, high-quality deliverables. This goal is not owned by one person, but rather the entire team.
Signs of a successful QHM
- Participation from everyone - When we first rolled QHM across teams, we thought that QEs will be leading the discussions. However, this was not the case at all. This proved that all team members (developers, especially) cared about their product and process quality and they wanted improvements. This kind of attitude is key to the next point.
- Team buy-in on action points - The team agrees on 1-3 things to improve before their next checkpoint. These action points are distributed amongst the team members (hint: it shouldn’t all be owned by the QE). These action points are incorporated to the team's roadmap or added to their backlog so these become project deliverables (instead of a side task).
- Positive trend - There are 2 positive changes (over time) that a team can have:
- The first is the visible change you see in the ratings. Below is a good example of how a team shows improvement as evidenced by the increase of “healthy” scores towards the fifth checkpoint.
- The second is how the team pushes itself to higher limits - The ratings per attribute shouldn’t only be the important data you should look at. It is the discussion points the team members have that you will want to pay closer attention to. For example, a team isn’t satisfied against the output for a specific attribute in checkpoint #1 and they believe they can do better in checkpoint #2. This is a positive sign of team maturity.
Should you use QHM as a metric?
Yes. However, it’s not as straightforward as counting “healthy”, “bit sick” and “unhealthy” ratings. As mentioned earlier, you will need to look at the quality of the conversation the team has during the QHM workshop. At Atlassian, we have collated all common issues across teams to identify problems that can be addressed at a higher level or solved through cross-team collaboration.
Also, there is no apples-to-apples comparison across teams. It is tempting to compare teams based on their health ratings. We cannot say that Team X is 'healthier' because it has 1 more 'healthy' score compared to Team Y. Instead, we should be looking for best practices that we can recommend to other teams, finding out why do some teams have a particular issue but others do not, etc.
Atlassian QEs highly recommend the Quality Health Monitor to:
- Gauge your team’s quality maturity - Discover team sentiment and next steps towards improvement. After a few more QHM iterations, you'll see team progress.
- Bring awareness of the QE team’s goals - The 8 attributes in the Quality Health Monitor represent these goals and this becomes transparent to the rest of the organization.
- Align goals in the QE team - Everyone can stay in sync and know exactly what to prioritize daily.
- Promote open culture - This is the nature of any health monitor. This workshop encourages diverse perspectives and is a safe place for everyone in the team to discuss a specific area.
- Align quality goals with team goals - We’re not asking a team to fix all issues that were surfaced from the QHM. The team should select the ones they need to address to help deliver their team goals.
- Share the responsibility of quality in the team - By involving the entire team in the decision-making process, you're empowering everyone to share responsibility.
Download our sample QHM and modify the attributes that best fit your team and organization.
Coming up next
From quality assurance to quality assistance
Learn how to transition from the traditional QA model to the new world of quality assistance. Keep reading...
On a related note...
Engineering higher quality through agile testing
There's still a need for manual testing–but not in the way you might think! Keep reading...
Put it into practice
Can't wait to get started?
Check out opportunities to join our global quality assistance team. Explore careers at Atlassian...