Hello there,
How is it going? Welcome to 8bitmen.com

This article is a comprehensive write-up on how to pick the right database for our app? How to pick a database which would serve our requirements best.

There are so many types of databases, relational, document-oriented, key-value, wide-column & so many more. How do we figure out which one would suit best for our app?

This article presents multiple database application use cases, with related products/tech & their real-life implementations, that helps us put a finger on one & say this is what I want. 

For all the articles in the pick the right tech stack series navigate here

Also, here the general guidelines to be kept in mind when picking on the technology stack for your app.

So, without any further ado. Let’s get on with it.

 

1. USE CASE:

Are Transactions & Data Consistency Important to you? Are You Writing a Stock Trading, Banking & Finance App?

Or Do You Need to Store Data Having a Lot Of Relationships? Are You Writing a Social Network Like Facebook?

If you have a use case that makes transactions, ACID, data consistency super important to you. Like when writing personal finance, stock trading or a banking app.

You might want to pick a Relational DB. Relational DBs shine when it comes to transactions & data consistency. They comply with the ACID rule, have been around for ages & are battle-tested.

Also, they have a larger community. Seasoned engineers on the tech are easily available, you don’t have to go too far looking for them.

If your data has a lot of relationships like which friends of yours live in a particular city? Which of your friend already ate at the restaurant you plan to visit today? Etc. There is nothing better than a relational database for storing this kind of data.

Relational databases are built to store relationships. They have been tried & tested & are used by big guns in the industry like Facebook as the main user-facing database.

Some of the Popular Relational Databases Used In the Industry Are:

MySQL – Open Source. Written in C, C++. Initial release 23 May 1995

Microsoft SQL Server – Proprietary RDBMS written by Microsoft in C, C++. Initial release April 24, 1989

PostgreSQL – Open Source. Written in C. Initial release 8 July 1996

Maria DB – Open Source fork of MySQL, Written in C, C++, Perl, Bash, Initial release 29 October 2009

Amazon Aurora – Proprietary hosted Relational Database Service by Amazon, Initial release October 2014

Google Spanner – Proprietary hosted Relational Database Service by Google

Some of the Real-life Implementations of the Tech Listed Below:

One of Australia’s largest financial company uses MySQL to run online trading.

Credorax runs it’s payment processing service using MySQL  

Facebook powers it’s social graph with MySQL & Memcache

 

2. USE CASE:

Is Your Data Semi-structured? Do You Need a Flexible Schema? Do You Need Fast Read & Writes?

Are you building a Real-time Feed? Live Sports App? Product Catalog or a Web-based Multiplayer Game?

If you have a use case which needs horizontal scalability, performant read & writes. You ain’t sure about the database schema when you start writing the app.

There is a possibility that things might change over time. You are in need of something flexible which you could change over time with minimum fuss. You need a document-oriented database.

Typical use cases of this type of database are real-time feeds in web apps, Live sports apps, writing product catalogues, Inventory Management, storing user comments or an online multiplayer game.

Document-oriented databases really stand out when we need performant read & write as it caters to CRUD Create Read Update Delete use cases. When there isn’t much logic involved. Just quick persistence & retrieval of data.

Some Of the Popular Document-Oriented Databases Available Are:

Mongo DB – A NoSQL Document Oriented database. Written in C++, JavaScript, Python and Go

Couch DB – An Open Source NoSQL Document Oriented database. Written in Erlang

Google Cloud Datastore – A Proprietary NoSQL Document Oriented database offered by Google

Amazon Document DB – A Proprietary NoSQL Document Oriented database offered by Amazon

Some Of the Real-life Implementations of the Tech Listed Below:

SEGA uses Mongo-DB to improve the experience for millions of mobile gamers

Coinbase scaled from 15k requests per min to 1.2 million requests per minute with MongoDB

 

3. USE CASE:

Do You Need to Visualize Your Data? Are You Writing a Recommendation Engine, An AI-based app, An App For Fraud Analysis, or Building a Knowledge, Social, or a Network graph?

If you have a use case where you need to visualize your data. By that I mean, you need an understanding on how your data, entities are related to each other. Like that in a social graph.

Think of that pinned board in detective movies where pins are pinned on a board connected via threads. It does help in visualizing how entities are related & how things fit together right?

A Graph database fits best in these kinds of scenarios. Ideal use cases of graph databases are building social, knowledge, network graphs. Writing AI-based apps, recommendation engines, fraud analysis app, storing genetic data etc.

Some Of the Popular Graph Databases Available Are:

Neo4J – A graph database written in Java.

Some of the Real-life Implementations of the Tech Listed Below:

Walmart shows product recommendations to its customers in real-time using Neo4J graph database

NASA uses Neo4J to store “lessons learned” data from their previous missions to educate the scientists & engineers.

 

4. USE CASE

Do You Need to Fetch Data, Without Much Processing, With Minimum Latency?

Do You Intend to Store User Sessions? Implement Caching? Create Leaderboards? Implement a Publish-Subscribe System?

If you have a use case where you need to fetch data real fast with minimum fuss, backend processing. Key value stores are pretty efficient in pulling off these scenarios where super-fast data fetch is the order of the day.

Ideal use cases are implementing caching, user state, sessions, managing real-time data. Implementing queues, creating Leaderboards in online games or web apps. Implementing a pub-sub system etc.

Some of the Popular Key-value Stores Available Are:

Redis – Source available Key-value database written in ANSI C

Hazelcast – Open source in-memory data grid written in Java

Riak – Distributed NoSQL Key value datastore written in Erlang

Voldemort – Distributed Key-value data store written in Java

Memcache – A general-purpose distributed memory caching system written in C

Amazon Dynamo DB – A proprietary key value, document database by Amazon

Some of the Real-life Implementations of the Tech Listed Below:

Inovonics uses Redis to drive real-time analytics on millions of sensor data

Microsoft uses Redis to handle the traffic spike on its platforms

Google cloud uses Memcache to implement caching

 

5. USE CASE 

Do You Intend to Manage Data in Real-time? Keep Track of Data Over a Period Of Time? For Instance, Ingesting Data In Real-time From Self-Driving Vehicles, Smart Home Systems, IoT Devices?

Do You Need to Store Data For Running Analytics, Monitoring? Are You Writing An Autonomous Trading Platform?

If you have a use case where you need to manage data in real-time & continually over a long period of time. A Time Series database is what you should be looking for.

Time series databases are built to deal with data, streaming in real-time. Typical use cases are fetching data from IoT devices. Managing data for running analytics & monitoring. Writing an autonomous trading platform which deals with changing stock prices in real-time.

Some Of the Popular Time-series Databases Available Are:

Influx DB – An open source time series database written in Go

Timescale DB – An open source time series SQL database

Some of the Real-life Implementations of the Tech Listed Below:

IBM uses Influx DB to run analytics for real-time cognitive fraud detection

Spiio uses Influx DB to remotely monitor vertical lining green walls & plant installations.

 

6. USE CASE

Do you Intend to Handle Big Data? Are You Developing A System to Ingest A Massive Amount Of Data For Analytics, Processing etc?

If you have a use case where you intend to grapple with massive amounts of data, technically Big Data. Pick a wide-column database.

Wide column databases are built to manage big data ensuring scalability, performance & high availability at the same time.

Popular Wide Column Databases Available Are:

 Apache Cassandra – An open source distributed wide column store written in Java

HBase – An open source distributed database modelled after Google’s Big table written in Java

Google Bigtable – A proprietary compressed, high-performance data storage system written in C++, Java, Python, Go & Ruby

Scylla DB – An open source distributed NoSQL datastore written in C++

Some of the Real-life Implementations of the Tech Listed Below:

Netflix uses Cassandra as the backend database for the streaming service

Adobe uses HBase for processing large amounts of data

 

7. USE CASE

Are You Building A Decentralized Blockchain Crypto-based App?

There are specialized data stores for blockchain. Which facilitate the development of decentralized apps.

Some of the Popular Ones Available Are:

BigChain DB – An immutable data storage with blockchain characteristics.

Amazon Quantum Ledger Database – A proprietary database by Amazon built for crypto & blockchain.

Some of the Real-life Implementations of the Tech Listed Below:

Resonate uses BigChain DB to create a music streaming platform which enables the music creators to own the music. And listeners to pay as the music is streamed by the creator.  

 

8. USE CASE

Are You Writing A Dedicated Mobile App For Your Business & Need A Database For That?

There are dedicated backend storage services available specifically to cater to the mobile app persistence requirements. Such as Google’s Real-time Firebase database.

It helps sync user data across all the client in real-time, provides offline support etc.

Besides backend services, there are Lite versions of popular databases available to be used with mobile devices.

Some Of the Popular Ones Available Are:

Firebase – A NoSQL cloud database by Google. Built for mobile apps.

Couchbase Lite – A full-featured embedded NoSQL database built for mobile devices

SQL Lite – An embedded relational database management system contained in a C library

Some of the Real-life Implementations of the Tech Listed Below:

One Football uses Firebase to power it’s mobile soccer app

 

9. USE CASE

Is Your Data Hierarchical In Nature? Does It Has A Tree Structure? Do You Want To Store An Organization’s Structure Or A File System?

Either of the document-oriented or the relational databases can be used to store hierarchical data.

Below Are Some Of the Good reads On How to Pull it Off

StackOverflow article on which database is best suited to store hierarchical data

Mongo DB article on data models tree structures

Options for storing hierarchical data in a relational database StackOverflow

 

10. USE CASE

Are You Building A System Which Involves Multiple Use Cases Stated Above?

In case you need several different kinds of databases, document, key-value, relational etc. to implement different use cases. Consider building a polyglot persistence model.

Polyglot persistence is the practice of using different data models to handle different data storage requirements. This is quite widespread in the industry & is practised by every service running online handling heavy traffic every single day.

If you are concerned about the operational overhead of managing so many databases. Consider using a multi-model database.

A multi-model database is designed to support multiple data models catering to several different use cases. A multi-model database would typically provide features of a document-oriented database, a relational database, a key-value store etc.

Some Of the Popular Multi-model Databases Available Are:

 Arango DB – A multi-model database written in C++, JavaScript. Supports graph, document-oriented, Key-value data models etc.

Couchbase Server – A multi-model open source database originally known as Membase. Written in C, C++, Erlang & Go.

Oracle Database – A proprietary multi-model database by Oracle. Written in Assembly language, C, C++

Orient DB – An open source multi-model database written in Java.

Some of the Real-life Implementations of the Tech Listed Below:

Popcorn uses OrientDB to scale its Social Movie Network

MapR uses polyglot persistence for its data platform

 

11. More On the Blog

How Hotstar scaled with 10.3 million concurrent users – An architectural insight

How Evernote migrated & scaled their cloud with Google Cloud Platform

What Database Does Twitter Use? – A Deep Dive

Why Use Cloud? How Is Cloud Computing Different from Traditional Computing?

A Thorough Insight into What is a Cloud Architect? And Why Should You Become One?

 

Well, Guys!! this is pretty much it. A few things…
If you liked the article, do let me know in the comments. Share it with your folks. If you think any particular use case is left out, do share your insights.

This doc is continually updated as the technology evolves.

For all the articles in the pick the right tech stack series navigate here

You can follow 8bitmen on social media, do subscribe to the browser notifications for updates on the new content on the blog.

I’ll see you in the next article
Until then…
Cheers