How Evernote Migrated & Scaled their Workload with Google Cloud Platform
Evernote is a leader when it comes to offering a service which enables us to save notes, capture our thoughts, project ideas, to-do lists, organize our data, also to collaborate with our teams online. The business has over 200 million happy customers.
In the recent past, it migrated its workload to the Google Cloud platform.
Before getting into the specifics,
Some of the Notable Stats:
Evernote has over 200 million customers around the globe.
It stores 5 billion user notes and 5 billion user uploaded attachments. That’s over 3.5 petabytes of data; Just so you can relate, this is equivalent of roughly 10 copies of every book ever published.
All of this workload was migrated to the cloud in just 70 days with top-notch planning & execution.
The present Evernote cloud infrastructure is more scalable, elastic & secure. The workload is powered by the Google Cloud infrastructure which enabled the Evernote engineering teams to focus on innovation, developing new features & not worrying about managing the underlying infrastructure.
The applications uptime, performance, monitoring is being assisted by Datadog. Which gives better visibility across the workload.
It enabled the devs to create custom dashboards to further enhance the visibility into the workload stats, metric from Google Compute Engine, Kubernetes & the App Engine.
Datadog is a Google Cloud partner, it’s a monitoring service for the applications powered by the cloud.
Reason for the Migration
Evernote originally managed its own on-prem servers and network. But with the increasing popularity of the business, scaling the infrastructure, maintenance, upgrade became challenging.
Dynamically scaling the infrastructure wasn’t as smooth as it’s on the cloud.
The infrastructure maintenance was eating up resources which should be better utilized innovating & writing new features. Focussing on what the customers want.
Migrating to the cloud, letting the experts handle the infrastructure would have definitely enabled them to move fast.
Core Modules of the Evernote’s Service
Let’s have a look at the core modules of the Evernote’s service
USER NOTE STORE
This service module stores the user’s notes. The service block is built on shards & every shard has a capacity of supporting over 300 thousand users.
A shard comprises of:
A client facing front-end service which runs on Tomcat
A relational MySQL based data storage layer
A Lucene search service which indexes user-generated content.
There are approx. 762 of these shards which comprehensively handle the entire customer base of Evernote.
This is a file storage layer which separately stores all the attachments which the users upload. This storage alone contains 206 servers. The attachments when uploaded are stored at two locations one locally & the other at a remote disaster recovery data centre.
USER DATA STORE
This is a central user database which contains the user information, manages the authentication & stuff.
Besides these, there are quite a number of additional services such as handwriting & text recognition, caching, batch processing, all these are powered by an additional 200 Linux servers.
Moving a workload with petabytes of data from an on-prem datacentre to the cloud is no easy feat. It requires a lot of care, focus & strategic planning. Though the entire migration was pulled it off in just 70 days.
A divide & conquer approach was followed in the migration in several different phases. Small chunks of workloads were migrated to the cloud & then validated & tested.
A few of the services such as the User Store, the Note Store which ran on Linux servers had a direct counterpart solution on the Google Cloud & were easy to move.
These services ran on physical Linux based datacentre & were migration to the Linux virtual machines on the Google Cloud.
The rest of the modules such as the User Attachment Store, Recognition service etc needed a significant custom modification to be moved over to the cloud.
The entire migration, movement of data had a thorough security audit. Data privacy, infrastructure security everything was accounted for.
The application is designed to run out of a single data centre serving the customers globally.
The workload is replicated in separate zones geographically to serve as a fallback in case of any unforeseen situation.
How Did Google Cloud Platform Help Evernote Scale?
After the migration to the cloud, Evernote’s engineering team is worry free of managing their infrastructure. They can now stay focussed on innovating & bringing new features to the market.
They’ve adopted the container technology. With state-of-the-art tools, they get a deeper insight into the production metrics. The Serverless compute has helped them with the costs.
The architecture has evolved from being a monolith to loosely coupled microservices. They are investing more on the serverless tech with Google Cloud Functions.
This write-up is a gist of a 5 part blog article published by Evernote about their migration to GCP
More On the Blog
Well, Guys!! This is pretty much it. If you liked the article. Do share it with your folks.
You can follow 8bitmen on social media. You can also subscribe to the browser notifications to stay notified of the new content on the blog.
I’ll see you in the next write-up
- Distributed Systems & Scalability #1 – Heroku Client Rate Throttling
- Zero to Software/Application Architect – Learning Track
- Java Full Stack Developer – The Complete Roadmap – Part 2 – Let’s Talk
- Java Full Stack Developer – The Complete Roadmap – Part 1 – Let’s Talk
- Best Handpicked Resources To Learn Software Architecture, Distributed Systems & System Design