Close

Negative velocity: how to lift the complexity limit

Signs your software team is navigating too much organizational complexity, and how to improve

Andrew Boyagi Headshot
Andrew Boyagi

Senior Evangelist


One of the most common goals for an engineering organization: deliver high-quality software, fast.

Take a look at your CIO or CTO’s vision statement and listen to them speak. Chances are, they're chasing some permutation of this goal. While it’s a common goal, there’s a wide spectrum between teams who attain this nirvana and teams stuck in software delivery purgatory. Some teams consistently drop new code into production with few incidents or negative customer impact, while others struggle with quarterly releases.

What causes this performance discrepancy?


Complexity in software delivery is the difference between delivering high-quality software, fast, and… not. It’s the difference between ringing the victory bell after a successful release every week and a disengaged software delivery team, frustrated that months of work on the latest release resulted in six new bugs and a rollback.

Compare the speed and quality at which startups ship new products and features with that of an established, large organization. For example, in the finance industry, fintech startups have chipped away at the market share of large, established banks over the past decade. Large banks commonly cite the unfair advantages of fintech startups, who operate with less regulatory oversight and no legacy, monolithic application estates to maintain. Smaller team size enables greater agility, and the ability to pivot based on customer needs. Essentially, fintechs don’t have the complexity of established banks, which enables them to move faster, with less risk. While it can slow software teams down, complexity isn’t always a bad thing.

Global network
related material

Tame software sprawl

three rings Icon
SEE SOLUTION

Manage your components with Compass

Complexity in software delivery


Complexity can be a good thing: it's extremely rewarding to solve gnarly problems. It motivates teams to rise to the challenge, tackle hard problems, and turn an industry on its head. Conversely, there is a point at which complexity is no longer about solving a hard problem, and has negative impacts on software teams.

Organizational complexity plays a key role in reducing the effectiveness of software teams. Collins dictionary defines complexity as, “the state of having many different parts connected or related to each other in a complicated way”. In practical terms, organizational complexity is the aggregate of information, dependencies, changes, other teams, tools, and requests, that software teams need to navigate when interfacing with the rest of the organization.

Higher levels of organizational complexity naturally make it more challenging to deliver high-quality software at speed, because teams spend more time navigating the organization than solving gnarly problems. Growing organizations soon learn that software teams reach a complexity limit – the amount of complexity teams are able to navigate before it impacts job satisfaction, and the quality and speed of software produced. It might seem logical, then, that reducing organizational complexity enables teams to focus on solving gnarly problems, delivering software faster, and with higher quality. Let’s explore why this isn’t necessarily the case.

Impact of complexity on a software team


Introducing a microservices architecture provides a good example of the impact complexity has on software teams. The definition of complexity is also a perfect description of a microservices architecture, “the state of having many different parts connected or related to each other in a complicated way”. It’s true that microservices allow teams to move independently, deliver faster, and safely scale systems - I’m a big fan of microservices - however, it undeniably adds significant complexity.

Let’s consider the effectiveness of software teams at Atlassian during the multi-year journey to adopting a microservices architecture.

At the beginning of the Atlassian microservices journey, DORA metrics looked great! Smaller chunks of code made it easier to test and deploy, allowing teams to safely move faster, and job satisfaction was high. During this phase, teams reaped the expected benefits of a microservices architecture. Though complexity increased, we saw no negative effect on teams.

beginning microservices journey

Fig 1. Beginning of microservices journey

Given the benefits, more of the organization began to adopt a microservices architecture, which naturally started to increase organizational complexity. More autonomous teams required more collaboration, and more microservices meant more dependencies. The pace of change skyrocketed - all the signs of software sprawl. The increased complexity caused software team effectiveness to flatten, indicated by a drop in change velocity, and cognitive load becoming an issue for software teams.

Fig 2. Team effectiveness flattens as complexity approaches the limit

Fig 2. Team effectiveness flattens as complexity approaches the limit

Without intervention, the organization eventually reached a complexity limit, and software team effectiveness declined. The benefits of speed, autonomy, and quality experienced at the beginning of the microservices journey reversed, which caused an understandable drop in developer satisfaction. These signs point to an organization that has reached its complexity limit. The software team spends more effort navigating organizational complexity than solving gnarly problems - not fun.

reach complexity limit

Fig 3. Team effectiveness regresses as the organization reaches the complexity limit

How to tell if you’re approaching the complexity limit


Reaching the complexity limit may feel inevitable, but there are some indicators that teams are approaching their limit. I’ll preface this by saying there is no absolute metric that tells you how close you are to the complexity limit, but these indicators may help you identify how close you are.

The clearest indicator that a team has reached the complexity limit is when they spend more time navigating organizational complexity than solving the gnarly problems they should be focused on. Trending the DORA lead time for changes (speed), and change failure rate (quality) metrics show if teams slow down or speed up over time. Though there are other factors that influence these metrics, they’re a good indicator of team effectiveness.

Developer satisfaction is another indicator of how much organizational complexity software teams are navigating. More than any other role profile, developers love spending their time solving gnarly problems, while equally loathing unnecessary tasks that get in the way. Low developer satisfaction is a good indication that organizational complexity is an issue for your software team.

Simplification is a losing strategy


When companies realize their teams are drowning in complexity, they often start a “simplification project”, aiming to restore simplicity in their organization. Simplification is a losing strategy for two reasons; complexity increases faster than any organization can simplify its environment, and these “simplification projects” operate in the very same complex environment they aim to simplify.

A common starting point for simplification is to reduce the number of applications by decommissioning or consolidating where possible. Decommissioning an application requires the team to understand all upstream and downstream dependencies, who the uses the application, how to replace the functionality, where and how the data will be archived or migrated, and deal with a host of other complications that arise with this type of work. Unfortunately, the significant investment in effort required to do this is not rewarded with an equally significant reduction in complexity. There is a limit to the amount of applications a company can decommission without impacting core business or user experience, but there is no limit to how many new software components engineers can create. In the 12 months it takes to decommission one application, it’s likely that hundreds of new microservices have been created. Since a healthy technology environment grows over time, it’s not practical to keep the environment to a fixed number of applications or software components to reduce complexity.

Simplification projects typically include reworking the organizational structure to remove complexity in communication flow. The least complicated organizational structures involve large hierarchical teams with all staff co-located. The most complicated organizational structures involve small, distributed, and autonomous teams. Conway’s law shows us that large hierarchical teams are likely to produce monolithic applications, while small distributed teams are likely to produce applications using modular architectures like microservices. Producing high-quality software at speed is enabled by modular architecture patterns such as a microservice architecture, which means the more complex organizational structure is more likely to lead to success. While “simplifying” the organizational structure makes it easier to understand, it is counterproductive to the ultimate goal of the simplification project.

Simplification is important and worthwhile, but is best built in as continuous improvement for software teams (and teams of teams) rather than a one-and-done activity. Simplification may delay reaching the complexity limit, but will not restore an organization to the fast-paced freedoms enjoyed in a startup environment.

Lifting the complexity limit


To return to software team effectiveness, organizations need to raise the complexity limit. Lifting the complexity limit essentially means increasing the amount of organizational complexity each team can navigate before it impacts job satisfaction, and the quality and speed at which the team ships software.

Platform engineering is an important concept in the quest to lift the complexity limit for an organization. Strong platform engineering teams focus on reducing the cognitive load of software teams by abstracting organization complexity from their day-to-day work. When implemented correctly, platform engineering enables teams to rebalance the bulk of effort towards solving gnarly problems, with less time spent navigating organizational complexity.

lifting the complexity limit

Fig 4. Lifting the complexity limit

Atlassian created Compass, a developer experience platform, for this very reason. Compass helps lift the complexity limit by making it easy for software teams to navigate organizational complexity through the component catalog, metrics and scorecards, and focus on creating a healthy engineering culture. The key callout here is organizational complexity did not reduce within Atlassian; in fact it continued to grow as more and more of the organization moved to a microservices architecture. We reduced the amount of time software teams spent navigating that complexity, which is the difference between a simplification project and lifting the complexity limit.

Atlassian has over 10,000 employees and over 17,000 software components, yet our software teams largely operate with startup-like freedom, shipping high quality software, fast. Our key to success? Lifting the complexity limit to improve software team effectiveness.

Here’s two actions to start lifting your complexity limit:

  • Track and assess your DORA metrics. How are the DORA metrics looking for your team? If you’re not tracking these already, the DORA metrics are provided out of the box with Compass.
  • Understand and assess developer satisfaction. How are the developers feeling in your software teams? Most organizations conduct employee satisfaction surveys. Ask for the results, broken down by functional area, to gain insight into developer satisfaction. Key questions include rating the following statements:
    • I am proud to ship
    • The amount of stress in my job is manageable 
    • I understand how my work contributes to company goals

Alternatively, Compass captures this information during the CheckOps ritual, where teams share how they felt about the last week, along with details on what could’ve gone better.

Lifting the complexity limit requires a combination of tools, processes, and rituals. A developer experience platform like Compass can help you understand system health, map dependences, and create ongoing rituals, helping to lift the complexity limit and unlocking the potential of software delivery teams in your organization.

Try Compass for free today.

Andrew Boyagi
Andrew Boyagi

Andrew is a Senior Evangelist within the Agile and DevOps team at Atlassian. With extensive experience in both software delivery and operations in enterprise organizations, Andrew provides a practical perspective on how teams and organizations can maximize the benefits of Agile and DevOps based on real-life experience. Andrew has also established and matured a platform engineering function within the Commonwealth Bank of Australia, one of Australia’s largest financial institutions, supporting 7,000 engineers. Andrew is passionate about unlocking the potential of software delivery teams and believes platform engineering is key to success in modern technology environments.

Away from work, Andrew has a goal to ride all 10 of the top 10 best motorcycle rides in the world. He lives in Sydney, Australia with his wife and two children.


Share this article

Recommended reading

Bookmark these resources to learn about types of DevOps teams, or for ongoing updates about DevOps at Atlassian.

Devops illustration

Compass community

Overcoming obstacles illustration

Tutorial: Create a component

Map illustration

Get started with Compass for free

Sign up for our DevOps newsletter

Thank you for signing up