We all have the tendency to avoid things that are going to be difficult – whether physical, logistical, or otherwise. And we do this knowing full well that if we just faced the tough things head-on, it would result in a much lower degree of total difficulty in the long run. (For example, I once went three years without seeing a dentist, and was so afraid of what might be revealed that I continued to avoid it for another four years.) Ahhh, humans. Gotta love us.
In the software world, the “dentist” facing so many teams right now are deployments and continuous delivery. In all likelihood, your team has some level of deploy automation going already. If you work with a legacy application and/or IT infrastructure, the automation probably consists of a collection of scripts started at the command line. While far better than no automation at all, this is still time consuming, and tends to dampen enthusiasm for frequent deploys. And in accordance with our “open company, no bullshit” policy, I will tell you that Atlassian teams are not immune to this.
But at some point, you have to summon some extra gumption, and just do what needs to be done.
Movin’ on up
In accordance with another policy, “build with heart and balance,” we are reconciling with the deployment dentist and moving Fisheye to a continuous delivery model.
Aside from there being so many other cool projects competing for the team’s time, Atlassian also maintains several Fisheye instances that popular open source projects like Hibernate, Findbugs, and Apache rely on every day. So the stakes for changing up the delivery pipeline are higher than with our other products, which get deployed only to internal “dogfooding” instances before shipping. Nonetheless, momentum for tackling this has been mounting for some time.
Being an unabashed Bamboo partisan, I was thrilled to hear that the new deployments capabilities in Bamboo were a “massive catalyst” in motivating this move, according to Fisheye & Crucible dev lead Nick Pellow.
Prior to this, deploying Fisheye to our internal environments and open source community instances involved tickets to the IT team, specialized one-off builds, and scripts triggered at the command line. It was error prone in several ways, and the end-to-end process typically took a couple days from request to completion. It’s not surprising then that the team deployed to our own dogfooding instance about every three weeks, and to the open source instances only about every three months. The dental truant in me can sooooooo identify. But, clearly, this is no way to operate in today’s fast-paced world of software development.
The Fisheye team enlisted a champion on the IT team to handle the infrastructure work needed to upgrade their deploy automation: moving SSH keys around, changing permissions on directories so Bamboo can write to them, and updating deploy scripts to accept runtime arguments that can be passed in by Bamboo as variables. With the foundation laid in the development, test, and dogfood environments, the team set about orchestrating the deployments in Bamboo.
The deploy needs to be performed on a build agent within our network running Linux. So the first steps in the deploy automation job set those parameters as requirements, which informs Bamboo of which build agents it is allowed to delegate the job to. Next, peripheral files needed for the deploy are checked out from a dedicated repository. Finally, Bamboo pulls in the build artifacts produced by Fisheye’s CI plan and executes the deploy script.
The team now automatically triggers a deploy to their dev environment with each successful CI build, and it takes all of two minutes (!) to execute. Deploys to the test environment are push button, and also take just a few minutes. A huge boost to agility, and they’re only getting started.
Right now, the team is working on deploys to our dogfood instance. Once those are rock solid, they can show the automated-deploy-fueled rapid turnaround love to the instances used by open source projects. Not only will our open source pals get access to new improvements the moment they are ready, but the Fisheye team will be able to establish a rapid feedback cycle with non-Atlassian end users, which make each release of the product shine that much brighter.
At the moment, deployments to all three environments use artifacts built on master. But with the ability to release artifacts from any branch, now available in Bamboo 5.2, the team’s next move is to make use of that for code reviews (which are done in Crucible prior to merging to master) and exploratory testing. The team already uses Bamboo’s plan branches feature to run CI on branches with every commit, and ultimately they want to extend that by spinning up an environments for each branch to deploy to.
If you’ve been nodding along because you’ve been avoiding the continuous delivery dentist too, I hope this story has inspired you to take the plunge. I’ll be back with a retrospective on Fisheye’s move to Bamboo deployments in a few months. In the meantime, take the first steps toward a clean bill of health and try out deployment projects in Bamboo for yourself.
PS: Haven’t checked out Fisheye? No time like the present!