If you’ve ever watched a pull request sit for hours because of a flaky test, you know how quickly test suites can turn from safety net into bottleneck. As teams grow, test suites tend to grow even faster because every new feature, bug fix, and regression adds more tests, while old or redundant tests are rarely cleaned up, so over time you end up running far more tests than are strictly needed for reliable feedback. Without the right tooling, it’s easy to end up with slow pipelines, noisy failures, and a backlog of flaky tests nobody has time to fix.

That’s why we’re introducing the open beta of a new Tests feature in Bitbucket Pipelines – a new way to organize, track, and optimize your tests directly where your builds run.

What is Tests?

Tests in Bitbucket Pipelines gives you a single place inside your repository to:

  • See the health of every test over time
  • Drill into detailed execution history for each test
  • Identify and quarantine flaky tests so they stop blocking your team
  • Automatically skip quarantined tests in your pipelines’s build script to keep builds fast and reliable

All from within the place you already use to review code and run your builds and powered by the JUnit-style XML reports you already generate today.

Test summary page

Turn Test Chaos Into Clarity

Once enabled, you’ll see a new Tests tab in your Bitbucket repository.

1. Test summaries: insight at a glance

The Test summaries view aggregates data for each individual test based on its recent runs, including:

  • Failure rate – how often this test fails
  • Average duration – how long it usually takes
  • Variance – how much its execution time fluctuates

With search and filters, you can instantly:

  • Find tests by name or prefix
  • Filter by labels
  • Filter by test state (for example, Ok, Flaky or Quarantine)

This makes it easy to spot:

  • Frequently failing tests that hurt developer confidence
  • Slow tests that drag out pipelines
  • Unstable tests with highly variable run times

2. Test executions: every run, fully traceable

Click any test in the summaries view to open its Test executions:

For each execution you’ll see:

  • Build – the build that ran the test
  • Commit – the commit associated with that run
  • Last executed – when it last ran
  • Execution duration – how long it took
  • Outcome – whether it passed or failed

This history helps you answer questions like:

  • “When did this start failing?”
  • “Is this failure tied to a particular change?”
  • “Has this test always been this slow?”
Test execution page

Controller Take Back Control From Flaky Tests

Flaky tests – those that sometimes pass, sometimes fail with no code changes – are one of the biggest drags on developer productivity and trust in CI.

With tests, you can classify problematic tests instead of letting them silently erode confidence.

Mark tests as flaky or quarantined

From the Tests UI, you can mark any test as:

  • Flaky – unstable and needing attention
  • Quarantined – unreliable enough that it shouldn’t block builds right now

Behind the scenes, Bitbucket maintains a test metadata file for your repository that tracks these classifications. Your pipelines can download this file and use it to decide what to run.

Enable the injection of the test metadata file – In the yaml, define it at the step level or at pipeline level.

pipelines:
  default:
    - step:
        name: Skip Quarantined Tests
        test-management:
          inject-classification: true

Example of instructing pipelines to skip quarantined, flaky tests 

pipelines:
  default:
    - step:
        name: Skip Quarantined Tests
        image: maven:3.8.4-openjdk-11
        test-management:
          inject-classification: true
        script:
          - echo "Parsing test-metadata.json to identify quarantined tests..."
          - apt-get update && apt-get install -y jq
          - QUARANTINED_TESTS=$(jq -r '.quarantinedTests[] | "!\(.class)#\(.name)"' "$BITBUCKET_TEST_METADATA_FILE_PATH" | paste -sd "," -)
          - mvn clean install -Dtest="$QUARANTINED_TESTS"

Keep builds moving by skipping quarantined tests

If you opt in, Bitbucket Pipelines can:

  1. Download the latest test metadata file into your build container.
  2. Let your build scripts read the file and skip quarantined tests automatically.

For example, in a Maven-based Java build, you can:

  • Parse the metadata file (using a tool like jq) to extract quarantined tests
  • Pass those tests into your mvn command so they’re excluded from the run

The result:

  • Fewer red builds caused by known-bad tests
  • Faster feedback loops for every pull request
  • Lower CI cost because you’re not re-running known failures over and over

You still see and track quarantined tests in the Tests tab, so they’re not forgotten – but they no longer hold your entire team hostage.

Works With the Tools You Already Use

Tests consumes JUnit-style XML test reports, which are supported by most popular frameworks.

Today, this includes (among others):

  • Java / JVM – JUnit, TestNG (via JUnit XML output), Maven Surefire
  • JavaScript / TypeScript – Jest, Mocha, Cypress, Playwright, TestCafe, WebdriverIO
  • Python – Pytest
  • .NET – NUnit

If your tests already produce JUnit-style XML, you’re ready to go. Otherwise, you can configure your framework to output in that format and Bitbucket will pick it up automatically.

Get Started in a Few Steps

If your workspace is on the Standard or Premium plans, you are automatically able to use these beta features by default. For repositories already using Bitbucket Pipelines, tests will be automatically be ingested if the test frameworks used are producing the JUnit-XML style reports in the steps. Test Summaries and Execution pages will automatically be loaded for last few days.

If you’re not already using Bitbucket Pipelines and want to try Tests:

  1. Enable Bitbucket Pipelines for your repository (if it isn’t already).
  2. Configure your test framework to generate JUnit-style XML reports or change the paths which are identified by the tests feature. For supported formats, examples, and configuration details, please refer documentation: Get started with tests in Pipelines | Bitbucket Cloud | Atlassian Support
  3. Open your repository and click the Tests tab.
  4. Explore Test summaries and drill into Test executions to see your current test health. To dive deep into how tags can help in organising tests, refer – View and add tags to your tests | Bitbucket Cloud | Atlassian Support and To understand how Bitbucket surfaces flaky behaviour and trends, read:
    Understand and manage flaky tests in Bitbucket Pipelines | Bitbucket Cloud | Atlassian Support
  5. Start marking flaky and quarantined tests to bring structure and clarity to your suite.
  6. (Optional) Opt in to using the test metadata file in your pipeline steps so you can automatically skip quarantined tests and keep builds green and fast. This works together with flaky detection and quarantining, more details can be found on Quarantining flaky tests | Bitbucket Cloud | Atlassian Support

To learn more about Tests in Bitbucket Pipelines overall, including concepts and additional use cases, visit: Testing | Bitbucket Cloud | Atlassian Support

Whats coming next?

Stay tuned—this is just the beginning. We’re continuing to invest in richer analytics and smarter automation so that managing your tests feels less like a chore and more like a strategic advantage.

Soon, we’ll also have more features to help you detect, and even fix, flaky tests using statistical algorithms and AI agents.

Share your feedback during the open beta

As we build out Tests, your feedback will help shape its future. Whether you have feedback, questions, or need help, our community thread is the go-to place to discuss your experience with Tests, report issues, or share ideas. Your input will directly influence the GA release and beyond, so let us know how Tests is working for you!