Amadeus shares strategies and tips to use when implementing Atlassian products at enterprise scale – including monitoring and scaling to Data Center.
This is a guest post written by Hugo Questroy, Senior Manager for CI/CD Pipeline at Amadeus
It’s not uncommon for technology projects at large companies to take months – even years! But our customer, Amadeus, did it in just a few months. How?
A few years ago, the Software Development Engineering group at Amadeus, a company with over four billion euros in annual revenue focused on providing technology to keep the travel sector moving, was asked to find a collaboration and development platform that could serve their entire company’s needs. Not only did the solution have to serve a wide variety of teams – including development, product analyst, project manager, QA engineer and various corporate functions, it also had to be up and running quickly. With over 15,000 employees in more than 70 countries, the task seemed near impossible at the time. How could the company meet each of its teams’ needs?
After some research, Amadeus decided to roll out multiple Atlassian Server products that are tightly integrated and built to work together – Bitbucket, Jira Software, Confluence, Hipchat, and Jira Core. After their incredible journey of implementing 5 products, here’s what they learned:
1. Users like Atlassian products
Amadeus was planning an entire internal promotion campaign to promote its newly acquired Atlassian products, but the team was pleasantly surprised to discover that Atlassian products didn’t require much internal promotion. Instead, Atlassian products experienced viral adoption, especially with Confluence. In the first month, more than 2,700 pages were created in Confluence. That was a huge early win!
2. Integrations are important, but can spiral out of control
Shortly after rolling out the products, the team received many requests for integrations with other internal tools within Amadeus’ ecosystem. A few examples include integrating Bitbucket with SonarQube for fast feedback in pull request, a Hipchat integration with systems status notifications for quick and targeted monitoring and alerting, and integrating Jira to synchronize with Amadeus’ internal ticket management system.
Amadeus wanted to satisfy its users, so it began adding these integrations and even allowed its teams to build their own integrations using the APIs. However, Amadeus quickly realized it had little visibility or control over integrations its users were creating or how they were being used. Today, Amadeus recommends implementing governance and controls for its users before rolling out integrations.
Amadeus uses SonarQube with Bitbucket for fast feedback in a pull request.
3. Plan for growth
Since Amadeus rolled out the Atlassian products, the number of Confluence users and pages created nearly doubled and this began to impact performance. In order to scale more effectively, Amadeus moved from Bitbucket Server to Bitbucket Data Center and Confluence Server to Confluence Data Center.
It is continuing to move its other key products to Data Center editions. By moving to Data Center, Amadeus was able to deploy multiple nodes, splitting the load across each of these nodes to reduce the potential for downtime or degraded performance.
4. Create a team to support and own your Atlassian products
Amadeus created a team that was global, connected, and accountable for the entire Atlassian suite including other tools integrated with its various Atlassian products. This team it built ended up as a virtual team – composed by members of different teams all around the world. Such a setup allows a follow-the-sun support, but also efficient knowledge sharing between members. Additionally, this team is able to support and maintain the whole set of tools in a unified way, sharing the same monitoring tools, processes, and SLAs.
5. Monitoring is essential to building a strategy that scales
In order to proactively identify potential performance issues, Amadeus set up a variety of monitoring systems. One of these monitoring tools was an infrastructure monitoring system that allowed them to monitor CPU consumption, network latency, memory usage, and other low-level probes. However, even with a healthy infrastructure, there can still be problems that impact the system and what users experience.
For example, if one of Amadeus’ LDAP servers went down, everything monitoring the infrastructure would not indicate an issue, but users would still be unable to access the system. To protect against these types of issues, Amadeus implemented further monitoring that it refers to as “functional monitoring”. Its functional monitoring system evaluates user-based scenarios like creating a Confluence page or sending a Hipchat message to make sure that users are not experiencing any issues with using or accessing any of the Atlassian products. Advanced monitoring and alerting, as well as investigation tooling, don’t make a system more stable. However, they limit the impact on end users by allowing Amadeus to be more proactive in preventing or handling potential issues.
Amadeus uses Dynatrace for monitoring.
6. Support your remote teams
Even after deploying Bitbucket Data Center, some users were still experiencing delays when cloning a repository. Amadeus is currently working with Atlassian to solve for this issue by implementing smart mirroring in Bitbucket. Smart mirrors are geographically distributed servers that are used for fetching code. These mirror servers operate as read-only copies of repositories in their Boston, Bangalore and other remote locations. These read-only copies in each location will be automatically updated by the primary Bitbucket instance in Nice, France. Users will now quickly see a reduction in delays.
7. Set up controls
In addition to this improved visibility, the team also needs increased confidence and control. Atlassian releases new updates to its products on a fairly frequent cadence, to give additional value to its users. To keep up with these release cycles, Amadeus is implementing things like performance tests to ensure smooth upgrades and confidence in the system. Amadeus is also focusing on controlling the openness that Atlassian products provide with things like plug-in governance and an API gateway. By controlling this openness, Amadeus will be able to make sure that users don’t abuse the system and impact other users’ abilities to successfully access and use the products.
8. Trust is key
At the end of the day, when introducing a new platform you must establish trust among your team that this solution will work for them, be reliable, and stable. Instead of thinking about internal promotion, think of ways that you can build trust by assuring your users that you have taken all circumstances into account to ensure the new product will support their needs – even as usage grows. That trust that you establish early on with your users will take you a long way in the future.
Since Amadeus initially purchased Atlassian products, the company has moved to Confluence and Bitbucket Data Center to support their quickly growing organization and its usage and will be moving to Jira Software Data Center and Hipchat Data Center later in the year. Amadeus continues optimize its long-term strategy for growth, always aiming to provide the best experience to its users.