Negative velocity: how to lift the complexity limit
One of the most common goals for an engineering organization: deliver high-quality software, fast.
Take a look at your CIO or CTO’s vision statement and listen to them speak. Chances are, they're chasing some permutation of this goal. While it’s a common goal, there’s a wide spectrum between teams who attain this nirvana and teams stuck in software delivery purgatory. Some teams consistently drop new code into production with few incidents or negative customer impact, while others struggle with quarterly releases.
One of the most common goals for an engineering organization: deliver high-quality software, fast.
Take a look at your CIO or CTO’s vision statement and listen to them speak. Chances are, they're chasing some permutation of this goal. While it’s a common goal, there’s a wide spectrum between teams who attain this nirvana and teams stuck in software delivery purgatory. Some teams consistently drop new code into production with few incidents or negative customer impact, while others struggle with quarterly releases.
What causes this performance discrepancy?
Complexity in software delivery is the difference between delivering high-quality software, fast, and… not. It’s the difference between ringing the victory bell after a successful release every week and a disengaged software delivery team, frustrated that months of work on the latest release resulted in six new bugs and a rollback.
Compare the speed and quality at which startups ship new products and features with that of an established, large organization. For example, in the finance industry, fintech startups have chipped away at the market share of large, established banks over the past decade. Large banks commonly cite the unfair advantages of fintech startups, who operate with less regulatory oversight and no legacy, monolithic application estates to maintain. Smaller team size enables greater agility, and the ability to pivot based on customer needs. Essentially, fintechs don’t have the complexity of established banks, which enables them to move faster, with less risk. While it can slow software teams down, complexity isn’t always a bad thing.
related material
Tame software sprawl
SEE SOLUTION
Manage your components with Compass
Complexity in software delivery
Complexity can be a good thing: it's extremely rewarding to solve gnarly problems. It motivates teams to rise to the challenge, tackle hard problems, and turn an industry on its head. Conversely, there is a point at which complexity is no longer about solving a hard problem, and has negative impacts on software teams.
Organizational complexity plays a key role in reducing the effectiveness of software teams. Collins dictionary defines complexity as, “the state of having many different parts connected or related to each other in a complicated way”. In practical terms, organizational complexity is the aggregate of information, dependencies, changes, other teams, tools, and requests, that software teams need to navigate when interfacing with the rest of the organization.
Higher levels of organizational complexity naturally make it more challenging to deliver high-quality software at speed, because teams spend more time navigating the organization than solving gnarly problems. Growing organizations soon learn that software teams reach a complexity limit – the amount of complexity teams are able to navigate before it impacts job satisfaction, and the quality and speed of software produced. It might seem logical, then, that reducing organizational complexity enables teams to focus on solving gnarly problems, delivering software faster, and with higher quality. Let’s explore why this isn’t necessarily the case.
How to tell if you’re approaching the complexity limit
Reaching the complexity limit may feel inevitable, but there are some indicators that teams are approaching their limit. I’ll preface this by saying there is no absolute metric that tells you how close you are to the complexity limit, but these indicators may help you identify how close you are.
The clearest indicator that a team has reached the complexity limit is when they spend more time navigating organizational complexity than solving the gnarly problems they should be focused on. Trending the DORA lead time for changes (speed), and change failure rate (quality) metrics show if teams slow down or speed up over time. Though there are other factors that influence these metrics, they’re a good indicator of team effectiveness.
Developer satisfaction is another indicator of how much organizational complexity software teams are navigating. More than any other role profile, developers love spending their time solving gnarly problems, while equally loathing unnecessary tasks that get in the way. Low developer satisfaction is a good indication that organizational complexity is an issue for your software team.
Best for managing microservices: Compass
Compass is an ideal server management tool. It simplifies handling microservices architectures by putting disconnected data about engineering work and teamwork together in one central, searchable location.
Compass' features include:
- Get full visibility into service details with relevant APIs, libraries, documentation, key health metrics, latest deployment activities, on-call schedules, and more.
- Document and track upstream and downstream dependencies and understand performance impact across teams and services.
- View all incidents, deployments, and other critical activities for a service and its dependencies in one place.
Best for CI/CD: Bitbucket Pipelines
CI/CD is an acronym for continuous integration and continuous deployment.
Bitbucket Pipelines is a CI tool that automates processes. It works right out of the box with Bitbucket, an Atlassian cloud-based version control system. It can use code to manage pipelines, letting users commit pipeline definitions and start builds fast. Bitbucket Pipelines also has CD features that allow you to deploy projects to live infrastructure.
Part of the CI/CD development process is to build microservices. Bitbucket Pipelines fosters efficiency by simplifying workflows and integrating with Bitbucket's cloud version control system.
Best for ITSM: Jira Service Management
Jira Service Management is an add-on for Jira Software, a microservices-based application that lets you control IT services, such as incident, problem, and change management. Jira Service Management’s ITSM features allow IT teams to provide excellent service. There are several reasons why that is the case:
- Flexibility: JSM's collaborative approach can help streamline service delivery processes.
- Automation: The automation suite can help automate recurring tasks.
- Integration: JSM integrates seamlessly with other Atlassian tools.
- Security: It encrypts all data in transit using TLS 1.2+.
- Scalability: JSM is an agile ITSM product that can scale up to the enterprise level.
Best for documentation: Confluence
Confluence is a collaborative documentation tool. It’s ideal for creating and sharing documentation, critical in microservices architectures. Confluence offers a wide range of Confluence templates for various setups, including those using Kubernetes and Docker, a microservices tool that helps developers build, deploy, and run containers.
Confluence templates include multiple features and apps to help you capture, distribute, and update your technical documentation. Also, with Confluence, you can centralize all your documentation in one place and grant access to users only to what they need.
Best for bug tracking: Jira Software
Jira Software excels at bug tracking and project management. It provides a platform to track, prioritize, and resolve bugs effectively through an easily navigable interface. Jira Software's bug-tracking features contribute significantly to successful microservices management. They also address the potential for software sprawl.
With Jira Service Management, Jira's capabilities extend to streamline IT service management, including incident, problem, and change management within microservices and monolithic architecture.
Best for monitoring and logging: Prometheus
Prometheus is an open-source tool developers use to manage microservices. It collects extensive metrics, including counters, gauges, histograms, and summaries, that comprehensively view the application's performance. Prometheus also assists in real-time troubleshooting by providing a comprehensive monitoring and alerting system that enables developers and IT teams to identify and resolve issues promptly.
Best for testing microservices APIs: Postman
The distributed nature of microservices architectures significantly hamper traditional testing methodologies. Testing the entire system becomes complex and time-consuming because each microservice is an independent component. This is where a specific microservices testing tool like Postman comes in handy.
Postman simplifies the process of testing microservices APIs. Developers love that it can automate testing, enabling faster and more accurate results.
Some of the ways Postman simplifies the process of testing microservices APIs include:
- Visual request builder: Postman's visual request builder makes it easy to construct HTTP requests without writing code.
- Request collections: Postman allows you to organize your API requests into collections, making it easy to group related requests and share them with other team members.
- Predefined tests: Postman provides a library of predefined tests that you can use to validate the responses from your API requests.
- Test scripts: Postman allows you to write test scripts using JavaScript, giving you more control over your tests and enabling you to automate complex testing scenarios.
How to tell if you’re approaching the complexity limit
Reaching the complexity limit may feel inevitable, but there are some indicators that teams are approaching their limit. I’ll preface this by saying there is no absolute metric that tells you how close you are to the complexity limit, but these indicators may help you identify how close you are.
The clearest indicator that a team has reached the complexity limit is when they spend more time navigating organizational complexity than solving the gnarly problems they should be focused on. Trending the DORA lead time for changes (speed), and change failure rate (quality) metrics show if teams slow down or speed up over time. Though there are other factors that influence these metrics, they’re a good indicator of team effectiveness.
Developer satisfaction is another indicator of how much organizational complexity software teams are navigating. More than any other role profile, developers love spending their time solving gnarly problems, while equally loathing unnecessary tasks that get in the way. Low developer satisfaction is a good indication that organizational complexity is an issue for your software team.
Lifting the complexity limit
Why is monitoring and testing important in microservices architecture?
To return to software team effectiveness, organizations need to raise the complexity limit. Lifting the complexity limit essentially means increasing the amount of organizational complexity each team can navigate before it impacts job satisfaction, and the quality and speed at which the team ships software.
Platform engineering is an important concept in the quest to lift the complexity limit for an organization. Strong platform engineering teams focus on reducing the cognitive load of software teams by abstracting organization complexity from their day-to-day work. When implemented correctly, platform engineering enables teams to rebalance the bulk of effort towards solving gnarly problems, with less time spent navigating organizational complexity.
What are some common challenges in monitoring microservices?
Atlassian created Compass, a developer experience platform, for this very reason. Compass helps lift the complexity limit by making it easy for software teams to navigate organizational complexity through the component catalog, metrics and scorecards, and focus on creating a healthy engineering culture. The key callout here is organizational complexity did not reduce within Atlassian; in fact it continued to grow as more and more of the organization moved to a microservices architecture. We reduced the amount of time software teams spent navigating that complexity, which is the difference between a simplification project and lifting the complexity limit.
Atlassian has over 10,000 employees and over 17,000 software components, yet our software teams largely operate with startup-like freedom, shipping high quality software, fast. Our key to success? Lifting the complexity limit to improve software team effectiveness.
Here’s two actions to start lifting your complexity limit:
- Track and assess your DORA metrics. How are the DORA metrics looking for your team? If you’re not tracking these already, the DORA metrics are provided out of the box with Compass.
- Understand and assess developer satisfaction. How are the developers feeling in your software teams? Most organizations conduct employee satisfaction surveys. Ask for the results, broken down by functional area, to gain insight into developer satisfaction. Key questions include rating the following statements:
- I am proud to ship
- The amount of stress in my job is manageable
- I understand how my work contributes to company goals
Alternatively, Compass captures this information during the CheckOps ritual, where teams share how they felt about the last week, along with details on what could’ve gone better.
Lifting the complexity limit requires a combination of tools, processes, and rituals. A developer experience platform like Compass can help you understand system health, map dependences, and create ongoing rituals, helping to lift the complexity limit and unlocking the potential of software delivery teams in your organization.
Try Compass for free today.
What are some best practices for monitoring and testing microservices?
To test and monitor microservices effectively, use monitoring data, set the proper alert levels, automate tests, set up continuous integration and delivery pipelines, and regularly test their performance and security. These methods ensure that your platforms work reliably.
Join the Atlassian Community for more Microservices articles and discussions.
Share this article
Next Topic
Recommended reading
Bookmark these resources to learn about types of DevOps teams, or for ongoing updates about DevOps at Atlassian.
Compass community
Tutorial: Create a component
