Hello there, how is it going?
Welcome to 8bitmen.com

This article is an in-depth write-up on Grafana – An open source tool for running analytics & monitoring our systems online. It contains answers to all our questions about it such as What is it? Why use it? Can I deploy it on-prem? Are there any companies using it already? How popular is it?

Also, I’ll share a bit of my industry experience with the tool.

So, without any further ado.
Let’s get started.

 

1. What is Grafana & What Is It Used For?

Grafana is an open source solution for running data analytics, pulling up metrics that make sense of the massive amount of data & to monitor our apps with the help of cool customizable dashboards.

Grafana connects with every possible data source, commonly referred to as databases such as Graphite, Prometheus, Influx DB, ElasticSearch, MySQL, PostgreSQL etc.

Grafana being an open source solution also enables us to write plugins from scratch for integration with several different data sources.

The tool helps us study, analyse & monitor data over a period of time, technically called time series analytics.

It helps us track the user behaviour, application behaviour, frequency of errors popping up in production or a pre-prod environment, type of errors popping up & the contextual scenarios by providing relative data.

A big upside of the project is it can be deployed on-prem by organizations which do not want their data to be streamed over to a vendor cloud for security & other reasons.

Over time this framework has gained a lot of popularity in the industry & is deployed by big gunssuch as PayPal, eBay, Intel & many more. I’ll talk about the industry use cases up ahead in the article.

Besides the core open source solution there are other two services offered by the Grafana team for businesses known as the Grafana Cloud & the Enterprise. What are they? More on that up ahead in the article.

Before that, let’s dig a little deeper into the functionality & the architectural flow of the tool with having an understanding of what dashboards are? & How do they work?

 

2. What Is a Grafana Dashboard?

Here is a snapshot of a Grafana dashboard, monitoring stuff.

8bitmen.com Grafana Dashboard

 

The dashboards pull data from the plugged-in data sources such as Graphite, Prometheus, Influx DB, ElasticSearch, MySQL, PostgreSQL etc.  These are a few of many data sources which Grafana supports by default.

The dashboards contain a gamut of visualization options such as geo maps, heat maps, histograms, all the variety of charts & graphs which a business typically requires to study data.

A dashboard contains several different individual panels on the grid. Each panel has different functionalities.

 

2.1 My Industry Experience with Grafana

In my former organization where I worked as a backend dev, we used Grafana for monitoring purposes, well naturally.

First on the pre-production environment to study the errors popping up & the server uptime.

The app instances were deployed as Docker containers managed by a docker swarm. There were times when the instances were down or a critical issue caused the system to crash. All of these scenarios were tracked on the Grafana dashboard. Which made our lives a lot easier.

The data was pulled from Prometheus which was plugged-in to the Grafana dashboard as a data source. Queries were fired from the dashboard with different expressions such as min, avg etc.

And Prometheus pulled data from cAdvisor. Have a look at the architectural flow.

8bitmen.com Grafana Prometheus cAdvisor Architecture dashboard monitoring

 

After the initial testing & monitoring was complete.

The tool was used to monitor the stuff happening when the app was live, in the production environment. Several pre-meditated checks were put in place & alarms were configured when they occurred. This helped us massively, in gaining an in-depth understanding of the system’s behaviour.

We also used past data, which we could track on the dashboard by filtering down by time range for planning out the future operations.

We also used Kibana for monitoring but that was primarily for tracking & studying logs.

 

3. What Are the Features Offered by Grafana?

Let’s go through the features offered by the open-source analytics dashboard framework.

This open-source framework takes care of all the analytics of our app. We can easily query, visualize, set up alerts, understand the data with the help of metrics.

The dashboard is pretty equipped, & continually evolving, to make sense of complex data. From displaying graphs to heatmaps, histograms, Geo maps. The tool has a plethora of visualization options to understand data as per our business requirements.

Alerts are set up & triggered like trip wires whenever an anticipated scenario occurs. These happenings can be notified on Slack or whatever communication platform the monitoring team uses.

Grafana has native support for approx. a dozen databases. And with many more, facilitated by respective plugins.

Either host it on-prem or any cloud platform of your choice.

It has built-in support for Graphite & expressions like add, filter, avg, min, max functions etc. to custom fetch data. What is Graphite? I’ll come to that.

It also has a built-in Influx DB, Prometheus, ElasticSearch, CloudWatch support. I’ll talk about all up ahead.

 

4. What is Grafana Cloud?

Grafana Cloud is a cloud-native, highly available, fast fully managed open SaaS Software as a Service metrics platform. Pretty helpful for those who do not want to take the load of hosting the solution on-prem & want to stay worry free of managing the entire deployment infrastructure.

It runs on Kubernetes clusters. The backend is Prometheus & Graphite compatible. So, we can either chose Grafana cloud instance or both.  For more information on navigate here.

 

5. What is Grafana Enterprise?

The Enterprise service comes with all the Grafana Cloud features plus premium plugins, data sources, premium support from the core team. We get response SLAs, trainings & a lot more. For more information visit

 

6. What Are Some of the Real-World Industry Use Cases of Grafana?

Grafana dashboards are deployed all over the industry be it Gaming, IoT, FinTech or E-Comm.

StackOverflow used the tool to enable their developers & site reliability teams create tailored dashboards to visualize data & optimize their server performance.

Digital Ocean uses Grafana to share visualization data between their teams & have in place a common visual data sharing platform.

These are some of the instances, why we love this tool. For further reading on the industry use cases here you go

 

By now I am pretty sure you have an idea what Grafana is & why use it?

Now let’s find out what are Graphite & Prometheus?

 

7. What Is Prometheus Grafana?

Prometheus is an open source data monitoring tool. The combination of Prometheus & Grafana is the de-facto tool combination in the industry for deploying a data visualization setup. Grafana dashboard is used for visualizing the data whereas the backend is powered by Prometheus.

Though Prometheus too has data visualization features & stuff. But still, Grafana is preferred for visualizing data. Queries are fired from the dashboard & the data is fetched from Prometheus.
It acts as a perfect open source data model for storing time series data.

 

8. What Is Graphite Grafana?

Graphite, again, is a monitoring tool. It facilitates storage & visualization of time series data. Ideally, Graphite is used as a data source for Grafana dashboard in a data monitoring setup.

Grafana has a pretty advanced Graphite query editor which enables us to interact with the data with the help of expressions & functions.

Should you wish to delve into the details. How Grafana, Graphite & Prometheus are different in terms of features they offer? This is a nice read.

 

9. Grafana Vs Kibana?

As I stated earlier. In my previous project, Kibana was primarily used for analyzing & monitoring logs. Kibana is the K in the ELK stack. The whole intention of writing Kibana by the ElasticSearch team was to have an effective tool to monitor logs. Just click around & track the context of exceptions occurring in prod instead of running Linux commands in the console to find them. It was a pain, believe me.

Well, on the other hand, Grafana is written as a generic monitoring solution for running monitoring & analytics on pretty much anything. Well, this is a very bird’s eye view of the difference between both the tools.

Read this, if you wish to get a deeper insight.

 

More on the Blog

What is Data Ingestion? How to Pick the Right Data Ingestion Tool?

An Insight Into Mattermost: The Enterprise Open Source Chat & Team Communication Tool

Designing a video search service on AWS – AWS Cloud Computing Architecture

An Insight into Bazaarvoice Scalable Architecture with Over 300 Million Visitors

Is My Data Safe in the Cloud? – A Deep Dive – All Your Questions Answered

How Evernote migrated & scaled their cloud with Google Cloud Platform

 

Well, Guys!! This is pretty much it. I hope you enjoyed reading the article.
I would love to know your views, thoughts if you have any. Do let me know in the comments.
If you liked the article do share it with your geek friends.

I’ll see you in the next article.
Until then…
Chao!!