HQ: Boston, USA
Offices: Boston & Tel Aviv (R&D)
Products: Bamboo, JIRA, Confluence & JIRA Studio
Barak Fargoun of Neocleus
We recently said to ourselves that we really need to write more case studies on customers using our software development tools. The stars were aligned when we got the following note from a devoted continuous integration (CI) customer:
We at Neocleus use Bamboo to build a very advanced testing lab for a type-1 hypervisor (bare-metal hypervisor). In our implementation, when you run a testing build plan in Bamboo, it does the following:
1. Deploys a fresh new image (native image – either Linux or Windows OS), on all of our lab machines (using fog for automatic image deployment)
2. After this, we deploy our product
3. Then, we are run a set of automatic testing tools, which runs our basic sanity tests for our product. (our tests are written in TestComplete)
4. Our automated testing programs have code to report their test results to a service we wrote. This service, then writes the results to Bamboo’s log, and to the J-Unit test files.
Overall, this system enables us to run a lot of bare-metal machine tests, see their tests online in Bamboo, and analyze those results using Bamboo. We did this by programming a lot of code which Bamboo simply runs and monitors.
I called Barak Fargoun, Chief Architect for Neocleus, who is based out of Tel Aviv, to ask him more about their set up. What follows is pure, CI awesomeness.
What does Neocleus do?
We are in the area of virtualization. We are extending open source virtualization products to enhance them by advancing their performance and user experience. This allows you to run a VM with your native graphics performance and native user experience using the devices you have on the machine.
How did you choose Bamboo?
We started using JIRA for issue tracking, and then we started using Confluence. After this, we started to examine Continuous Integration (CI) tools. It was pretty obvious we wanted to go with another Atlassian product. It was easy to integrate and bring in Bamboo rather than another CI tool.
What advice would you give a smaller dev shop considering Bamboo?
I’d recommend starting to use Bamboo from the first day of development. It took our managers a while to even think about investment in automated tests. Once we got Bamboo, it took us a few weeks to write the tests, and has since increased our productivity 2-3 times more than before. Each commit is being tested and built every night, all the time. You commit your change, and at night it runs 5-6 hours of testing on about 6 machines. In our lab it is very scalable because you just bring in a new machine, connect it, configure it and then Bamboo runs it every night. We don’t have to deploy it for QA, so it saves us lots of time.
Can you tell me about your Bamboo set-up?
We have about 6 people using it on a daily basis. The source control and test automation tools we use are SVN and some Mercurial because of our open source projects. We have 5 build agents, none of which are in the cloud.
We have also not used Bamboo gadgets in other products yet. We have just started playing with the idea to show an executive summary of the current status of the code. We want to prepare basic Confluence pages as executive summaries of the source based on the Bamboo gadgets. Basically, Bamboo is a different unit—currently, the only connection is to display the overall status of the code in a Confluence page. People can visit it once a day to see the source code status.
We have been pleasantly surprised with the capabilities of agents which allowed us to configure different build agents with different capabilites. We used the capabilities of the agents instead of different test machines. Regarding notifications, we prefer to get email. We use Bamboo for CI and to deploy and release software. To perform the release build, we do it with a special build plan in Bamboo, then use the artifacts and just download it.
What sorts of builds does your team run?
We have a few—per each commit we have a sanity build which checks the tree is building correctly. We also run basic unit tests to make sure all unity tests are functioning and working OK. Each day we have about 2-3 nightly builds that are building full products and testing them which takes 4-5 hours each. There are also manual builds which a user can trigger if they, for example, start to develop a major feature in their own branch. They basically just modify or create a build plan to work on their specific branch and then run which ever test they want to run manually.
Can you please expand on your advance use of Bamboo?
We wrote a full testing framework in Python which involves other open source projects. For background, our software needs to be tested on bare-metal machines, so we can’t use a singluar VMware server with 20 different VMs to test our code. We want to test our product (bare-metal hypervisor) on different physical machines (e.g.: Dell, Lenovo, etc.) every night, so we will get the best platform coverage we can.
We use an open source tool called FOG as the image deployment server. When we program in it using our very own library which we wrote, it allows us to deploy a machine remotely. The machine reboots and deploys a clean image. This all happens after the build plan finishes building the product using Bamboo, and then it switches to test mode.
After we deploy the fresh new image we selected (in the build plan configuration parameters), we install the product we just finished building, then go to the automated ‘post-integration’ tests, which is basically a big set of tests. These run overall tests of the system. Each one of the machines has code allowing it to post test results to Bamboo in realtime. They send simple network messages to the Bamboo server. Using our own server daemon, which we wrote and deployed on the Bamboo server, these network messages are translated to test results and are reported to Bamboo (using the J-Unit format for tests). In the Bamboo messages log, you can track everything. When the build plan finishes, you have all the tests results: deploying an image, installing the product, post-integration tests, etc.