Today at FullContact we are proud to launch FullContact for iOS. The truth is, it’s been a significant challenge to get here.
Roughly a year ago, we ran into a pretty common scenario in our industry. We had a platform that was holding us back. Our first attempt at an address book platform a.k.a Address Book 1.0, exhibited serious warning signs of an impaired architecture: We were routinely introducing technical debt to move forward, and delivering quality code at a predictable pace was nearly impossible.
It was pretty clear: the biggest challenge in front of us was a monolithic core application built on Grails that had become extremely painful to develop in. Grails wasn’t the entire problem, we simply allowed it to grow to an unsustainable size. There were other equally costly sins:
- We had large quantities of unit-tests but very few integration tests – a big problem on its own when you have many moving parts.
- We had not paid enough attention to monitoring and tooling; tracking down issues was extremely time-consuming.
- The storage layer was reaching it’s limits with the sheer volume of data we were storing
- A combination of Groovy/Java nullified the advantages of both – refactorability of statically typed Java and agility of dynamically typed Groovy
It was obvious that without some fundamental changes we simply could not move forward. So, last April, we deemed the entire architecture as sacrificial (1).
Seismic shifts in architecture ironically come with a flashing bright red warning sign (2) – anybody who has ever done a big refactoring will tell you that it’s harder than you anticipate, and the likelihood of failing is probable.
Nevertheless the whole company bought in – if we didn’t invest in getting the fundamentals solid then we would never be able to move fast enough in future. With everybody on board we set out for The Grind – a rewrite/refactor that took quite a bit of time, patience and energy but was ultimately worth it.
The obvious place to start was breaking down the big monolithic Grails application into smaller services (a.k.a micro-services architecture). There were a few important questions we still had to answer: What technologies did we want to utilize? What did we want to refactor vs rewrite? How do we make sure the refactoring doesn’t break things? Most importantly, what we will name the new services?
At this point FullContact is deeply invested into the JVM platform. We have unified deployment, monitoring and our awesome devops team keeps JVM services running smoothly at all times. So sticking with our existing JVM platform for new services was an easy choice.
Next… the fun decision, choosing a language. We decided to remove Groovy from the equation as Java 8 had introduced the things that made Groovy so attractive. We also wanted to take the opportunity to shift to a more functional programming style and Java is less than perfect for that. Luckily there are now two solid functional languages on the JVM platform – Scala and Clojure. Ultimately we choose Clojure for it’s core value: simplicity. Java still has important place in our stack – not everything fits functional programming style and Java’s static typing is very beneficial in certain cases.
With our weapon of choice armed and loaded, It was onto the refactor vs rewrite debate. Surprisingly for us, it wasn’t much of one. We made a simple distinction, our core services for storing and retrieving contacts would be refactored into Java based micro-services. All other ancillary services would be rewritten with Clojure and backed by any other technology we deemed necessary to make them successful.
Last but not least, we didn’t want to follow Facebook’s mantra “Move fast and break things”. Our mantra is “Be awesome with people”. Be awesome by not breaking things. Be awesome by making sure our customer’s data is always safe and secure. The answer was meticulously automating integration testing. Instead of spending time unit-testing the smallest functions, we made sure that all our API endpoints and cross system behaviour have complete integration test coverage. This made rewriting code much more relaxing.
We can gladly say we’ve endured “The Grind”, and paid for our costly sins. Now that we’re through, we’ll be sharing (in great detail) everything we’ve learned in an upcoming series of technical blog posts. The following topics include: Breaking up the monolith (and you can too!), You always had me at Integration Tests, Devops? More like Awesomops, Adventure Time Naming Schemes and Where will Ms. Frizzle take us next.