Back in January we introduced smart mirroring for Bitbucket Data Center to help distributed teams by reducing clone times (as well as your list of excuses for taking a coffee break – sorry about that). At the same time, we added Git Large File Storage (LFS) for teams who need to work with images, videos, and other large media assets.
Both these features support using Git at scale, but in different ways. Smart mirroring helps teams spread across different locations, while Git LFS helps teams who need to manage large files.
Both are great on their own. But imagine what they could do for distributed development teams if used together…? Well imagine no more! We’ve combined support for Git LFS with smart mirroring in Bitbucket Data Center 4.5.
Smart mirroring and Git LFS in the wild
Our team has engineers all over the world, often contributing to the same repository. We wondered how one such repository, at 5GB in size, would perform when cloned between our Sydney and San Francisco offices using smart mirroring. A typical clone takes over an hour. But we found that using a local mirror cuts this time down to a mere 5 minutes (which is juuuuust enough time to run out for a flat white, Australia’s unofficially official coffee drink).
We also knew this same repository contained binary files that slowed down every clone operation. By moving the binaries to Git LFS, we were able to improve geographically remote cloning (i.e., no local mirror involved) of the new LFS-enabled repository down to 5 minutes as well.
5 + 5 = …1?
That’s when we got really excited. Giddy, even. We had to try the two features in combination.
To review, we started with a 5GB repo. Then trimmed it down using Git LFS. Then mirrored it between servers located in Sydney and San Francisco (a distance of 4,205 miles, according to my frequent flyer records) using smart mirroring.
These two changes took us from an original clone time of over an hour to a clone time of just over one minute.
A 5GB Git repo cloned from Syndey to San Francisco in under a minute. Yes: you read that right.
Here’s how it works. Mirroring of Git LFS objects is performed on-demand. That is, when a client requests a download of a Git LFS object from a mirror server, the object will be streamed from the upstream node only if it’s not already available on the mirror. Subsequent downloads of the same object will be downloaded directly from a copy stored on the mirror. It’s a match made in software heaven.
Don’t take my word for it
Still not convinced? Good. We welcome your skepticism! Upgrade to Bitbucket Data Center 4.5, or try it free, and find out for yourself.