Our Approach to Vulnerability Management

Our Approach to Handling Security Vulnerabilities @ Atlassian

Atlassian recognizes that, at some level, security vulnerabilities are an inherent part of any software development process. However, we are constantly striving to reduce both the severity of and frequency with which vulnerabilities arise in our own products and services.

To that end, we have in place a multi-faceted approach to vulnerability management that relies on a combination of both automated and manual processes. We believe that this is the most effective way to limit the chance of vulnerabilities “slipping through the cracks” and going undetected for an extended period of time.

In this paper, we provide an overview of how we go about managing vulnerabilities in our products and infrastructure, and how we’re constantly evolving that approach by incorporating the latest tools, methods and thinking to ensure our handling of vulnerabilities remains effective into the future.

An overview of our vulnerability identification and resolution process

We have a methodical process for identifying, tracking and resolving vulnerabilities, regardless of type.

Continuous asset discovery and attribution

Continuous internal asset discovery - We use in-house built system to inventory all of our EC2 and Load Balancer AWS Asset using AWSConfig and attribute it to the correct owner. We retain one year's worth of assets totaling 50-60 million assets.

Continuous external asset discovery - We use Assetnote to perform continuous asset discovery and security analysis on our external perimeter.

Identifying vulnerabilities

We use a range of best-of-breed vulnerability detection tools that are run regularly across our products and infrastructure to automatically scan for and identify vulnerabilities. This includes Atlassian Cloud & Server products, Docker application images, internal, mobile and third party applications, as well as our infrastructure both on premises and in our cloud. These tools automatically scan for and identify vulnerabilities that exist, and include:

• Network scans – We currently use Nexpose as our primary vulnerability management tool of choice, which helps us to identify active services, open ports and applications running across our environment, as well as any vulnerabilities that exist at the network level;
• Container image scans – We use Docker containers for deploying many of our applications and we conduct a full security scan that conducts a deep inspection of the contents of those containers anytime they are deployed into our production or pre-production environments. We do this using a tool called Snyk. More detail is provided later in this page.
• Open source dependency scans – We use SourceClear to identify any vulnerabilities that may exist in open-source or third party code our developers may be using. More detail is provided later in this paper.
• AWS configuration monitoring - We use Trend Micro Cloud One - Conformity to provide continuous configuration monitoring against established baselines for our AWS environments.

We are continually reviewing the latest tools available and adding them to the suite we use if we believe they will enhance our vulnerability detection capabilities.

We also have a range of additional avenues that we use to identify vulnerabilities in combination with the automated scans we run. These include:

Our Bug Bounty Program – We make use of Bugcrowd to run our Bug Bounty Program. Bugcrowd provide us with access to an expert, trusted community consisting of tens of thousands of cyber security researchers who are constantly testing our products and reporting back any vulnerabilities they find. Our Bug Bounty program has been recognized as the best in the industry in 2018 and 2019.

Customer & user reports – Users of our products can report any bugs they encounter at any time via Atlassian Support. We will then work with them to collect all necessary details so the vulnerability can be flagged internally and fixed (subject to validation to ensure that the vulnerability is real, and not a false positive). This also includes Atlassian staff, who can raise any issues they observe within our products (either externally, or internally) directly with the Security team, or by raising a support ticket.

External penetration testing - We use specialist security consulting firms to conduct white-box, code-assisted penetration tests on high risk products and infrastructure. See “Our approach to external security testing” for more detail.

Atlassian’s Product Security team - We complete targeted code reviews, both manual and tools-assisted, and work closely with our product development teams to enhance their ability to self-detect and resolve vulnerabilities before the code reaches us.

Atlassian’s Red team – We have an internal red team whose role is to simulate the role of adversaries attempting to identify and exploit vulnerabilities that exist within our systems, processes, and environments, so that we can ensure they are identified and addressed as promptly as possible.

Tracking and resolving vulnerabilities

To maximize the efficiency and effectiveness of our vulnerability management program, we integrate the processes we use to identify vulnerabilities with a centralized internal ticketing and escalation system. Specifically, regardless of whether a vulnerability is identified through one of our scanning tools or one of the other avenues we have discussed above, a dedicated ticket is created for each vulnerability and is assigned to the relevant product team for resolution in accordance with the SLOs we have published in our Security Bug Fix Policy.

Our security team provides oversight of this process and works with product and infrastructure teams to ensure accuracy of vulnerabilities, answer remediation questions, and ensure all vulnerabilities are resolved in compliance with our SLO commitments.

Once a fix for a vulnerability is developed, it is tested thoroughly and then, in the case of our cloud products, incorporated into our CI/CD pipeline for deployment. For server and data center products, fixes are rolled into a new release and deployed with other fixes on a regular bases in accordance with our standard release cadence. Vulnerability tickets from scanning tools are automatically closed when subsequent re-scans do not find the vulnerability. Vulnerability tickets from manual findings are closed by product, infrastructure, or security team members when the fix has been deployed to production.

Preventing vulnerabilities during the development process

Container image scans

Atlassian deploys the bulk of its applications using Docker container images. Docker containers provide a packaged, self-contained environment consisting of relevant system libraries, tools, configuration settings and any other dependencies required so that our products are able to run regardless of individual machine configuration parameters. The container effectively provides a layer of abstraction, decoupling the software code from the underlying infrastructure so that our products can work without issue across different machines.

While containers offer great benefits for our developers and customers in terms of being able to deploy code that can be used in a variety of environments, they can be a source of security vulnerabilities if the contents of the images consist of out-of-date or otherwise insecure libraries or components.

To address this, Atlassian integrates a full container security scanning process into our CI/CD pipeline for any containers that are deployed into our development, staging or production environments. We use the Snyk open source engine for this purpose. Snyk provides a set of tools that undertakes a deep inspection of any container images that are deployed by our developers. This includes a detailed analysis of those images to identify the various components they contain (including operating system and application packages, third party libraries as well as configuration files).

Open source dependencies

While it's important to find and fix vulnerabilities in our own code, our products and services also rely on numerous open source libraries. It is therefore equally critical that we are aware of what libraries we're using and that they're up to date with the latest security bug fixes. We use a tool called SourceClear to assist us with this. SourceClear provides a scanner that can identify dependencies in any of our software builds and can then compare these libraries to a database of known security vulnerabilities.

Any identified vulnerabilities, whether found via Snyk or SourceClear, are then raised automatically via a formal Jira ticket with the relevant product team in accordance with the vulnerability management process we described earlier on this page.

Other initiatives that we use to help combat vulnerabilities

So far in this paper, we’ve largely described steps we take to manage vulnerabilities at the ‘back end’ – i.e., what we do to address a vulnerability that is identified in our products or platforms. However, we are constantly striving to reduce the frequency with which they arise in the first place. To this end, we have incorporated some unique initiatives at the ‘front end’ of our development process to ensure that our products are built from the ground up with security in mind.

Security Champions Program

In 2017 we started rolling out a Security Champions program across Atlassian to embed security leaders within every one of our product and service teams. Our Champions are also provided with dedicated training to help them understand and identify application security vulnerabilities, as well as processes for writing secure code. The goal of this program is to have a dedicated Champion within each team who assumes responsibility for promulgating key security messages amongst fellow team members and raises any security issues with our central security team to facilitate improved communication flows.

Atlassian’s Security Champions meet on a monthly basis to share tools and knowledge around the latest security issues and challenges they’re facing so that all our teams can benefit. The ultimate aim is to use the Champions program as a springboard for enabling security to form an even more integral part of our culture. We believe that this kind of approach is a key pillar in our approach to minimizing vulnerabilities.

Product security engineers

No discussion of vulnerability management would be complete without explaining the key role our product security engineers have in both ironing out bugs, and designing better irons.

Our product security engineers perform the initial triage on newly reported vulnerabilities and collaborate with our product engineering teams to identify the best fix for the issue. Our product security engineers are subject matter experts in application security and are distributed globally so that they can most effectively collaborate with our product engineers as needed.

Our security engineers has both pro-active and re-active security roles in relation to their assigned product, including but not limited to:

• Reviewing and analysing up to date threat models for new and emergent risks
• Reviewing and analysing the security of new features
• Performing manual code review
• Performing penetration testing
• Performing platform and architectural review
• Tracking major activities related to projects and providing guidance when necessary
• Triaging, filing, rewarding, and ensuring the timely resolution of issues reported via our bug bounty
• Writing new automation and maintaining existing automation and tooling to maximise coverage and efficiency

Security scorecards

With the data collected from the systems described in this paper, we are able to benchmark teams and products against each other to pro-actively identify areas for improvement.

In summary

Atlassian has a multi-faceted approach to vulnerability management across our products and platforms that uses a combination of best-of-breed scanning tools, our bug bounty program and a range of other mechanisms that are constantly evolving to ensure we identify and resolve vulnerabilities that arise as quickly as possible, and minimize their frequency in the first place.

Want to dig deeper?

There are a range of other resources we’ve referred to in this paper or which you can otherwise consult to get more information about our approach to vulnerability management, as well as security more generally.

• Atlassian Trust Center
• Atlassian Bug Bounty
• Security Bug Fix Policy
• Approach to External Security Testing
• Continuous Delivery
• Security Advisories
  

On this page, when we refer to vulnerabilities this can be used interchangeably with ‘bugs’, which is the term we use in our separate paper on Our Approach to Security Testing.