As part of Atlassian’s Internal Systems team, I’ve been working on a new internal application called Scout. It’s a Grails app which aggregates data from a whole swag of different systems, which provided us with a few GORM-y challenges. GORM is one of my favourite bits about Grails – it’s how Grails does object relational mapping, and it’s awesome. Grails domain classes are much terser than Java POJOs, and they make CRUD a breeze (especially the R, which I find to be a bit of a pain).
Challenge One: Multiple Data Sources
While we did try to use application APIs wherever possible, for application speed and ease of development, we ended up needing to bypass some applications and query their database directly. Out of the box, Grails works with one data source, configured on a per-environment basis in DataSource.groovy. While this works for for most applications, Scout needed to access multiple databases.
Enter the Grails Datasources Plugin. This plugin made it possible for us to map each domain class to a different datasource. As with most Grails plugins, it was exceedingly simple to get it up and running. The plugin allows its datasources to be configured per-environment, equivalent to the default Grails behaviour, which was vital for us.
We did encounter a last minute bug where the production datasources were not being associated with the correct datasource, but fortunately the bug had already been discussed on the plugin author’s blog and it was a two line fix in one groovy file.
One unexpected bonus of this plugin is that it allowed us to use read-only datasources. Scout never needs to update another application’s database and it’s great that we were able to enforce this at the datasource level (in addition, of course, to restricting the permissions of the database user Scout uses).
Challenge Two: Legacy Databases
GORM works best when it is mapping to a completely new database, but in Scout’s case, we were stuck with legacy database schemas.
Mapping the existing tables onto Grails domain classes meant that we would have been tied quite strongly to the existing database – one table would match up with one Grails domain class. Renaming or changing columns would require break to Scout, resulting in application downtime until a fix + redeployment. I also happen to favour simplified domain classes over a complex set of relationships, where possible. It makes application maintenance a lot easier and lowers the bar of grokitude for the app.
We decided instead to map the domains classes to views. Using views helps us minimize the amount of domain classes we needed. The SQL to create the view is stored as a comment with the corresponding domain class, and if a database change breaks the view, all we have to do is re-create it and Scout will still work.
We didn’t have any problems mapping Grails domain classes to views – remember that the application only needs to read data so we didn’t have to worry about doing updates – though one trick we did rely on was in one table the column we had to use for the id was not unique. This wasn’t a problem for GORM because we weren’t doing updates and we don’t use the id for anything in the application.
Challenge Three: MySQL v4.0
One of the applications uses MySQL, where our other applications use Postgres. We made the classic mistake of developing against a different version of the database than what’s in production. Development, v5.0. Production, v4.0. What’s one of the big differences between the two? Views.
Arrrrrgh! (Not to mention we only found this out when we began deployment).
We didn’t want to re-write all the places where we used the domain classes that map to that one application’s classes (I’m also still hoping that we’ll upgrade the database soon-ish). I wanted a solution that we could just rip out at the right time.
Because Scout just aggregates information, all we had to do was make sure the dynamic findAllBy methods looked in the right place. This turned out to be a two step process. First, we implemented one domain class per table we were querying (rather than one domain class per view), then in the original domain classes, we overrode the specific findAllBy methods that Scout was calling.
For instance, searching through the codebase, we found that the we used two findAllBy methods on the ScoutMessage. We just implemented those methods:
public static Collection findAllByEmailLike(String email);
public static Collection findAllByEmail(String email);
These two methods use the dynamic methods on the new domain classes to bring back instances of those classes, which we then converted into the ScoutMessage objecta. This saved us a ton of work refactoring the code that worked with the ScoutMessage object. We were also able to return multiple domain objects per row from one findAll query which saved us a lot of time re-querying for information stored across two tables. We were able to implement this solution in a couple of hours which meant it didn’t impact our production deployment timeline.
This solution is brittle – it will be broken if we call any other dynamic method on ScoutMessage and it means we’re more susceptible to a database schema change. It is unlikely, however, that the database schema will change, and this solution is much better than not being able to search that application at all.
So, using Grails to develop Scout saved us a lot of time – we spent most of the development time on making sure that the search results were returned in a timely manner and on preventing information overload in the display. GORM saved us a lot of time implementing POJOs and custom DAO calls which left us plenty of time to finesse the view and to make sure we were writing reusable, extensible code.