Grafana (visualization)

Open-source server to display metrics for DevSecOps and others

Overview

This article are my notes about Grafana, marketed at Grafana.com, with documentation at https://grafana.com/docs/grafana/latest

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

My contribution here is logical sequencing for a deep yet concise hands-on tutorial.

Grafana was as open-source in 2014 at https://github.com/Grafana/Grafana. It is one of the most popular open source projects on GitHub.

The original developer, Torkel Ödegaard (@Grafana@torkelo), formerly at eBay Stockholm, Sweden, spoke Dec. 19, 2014 in San Francisco six months after release.

Grafana is written in Golang.
[21:53] “I work on this full time, mostly paid by my own savings. But I have sponsors in Squarespace and Dreamhost”.

Install

See https://logz.io/blog/grafana-tutorial/

https://www.grafana.com/download

Grafana’s releases are sematically tagged.

There is an Enterprise Edition because the organization makes money by selling its Enterprise edition and cloud services. See https://grafana.com/docs/grafana-cloud/quickstart/logs_agent_linuxnode/

Enterprise edition

https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/

Their Install Guide says “Open Source - Functionally identical to the enterprise version, but you will need to download the enterprise version if you want enterprise features”

Grafana Enterprise adds enterprise data sources, advanced authentication options (LDAP, SAML), more permission controls, 24x7x365 support, and training from the core Grafana team.

There is no installer for MacOS Enterprise, so run the Docker image on your Mac.

Raspberry Pi

There is an installer for ARM chips (Raspberry Pi). See https://www.hackster.io/search?i=projects&q=grafana

Releases
Details about each release are at:

https://github.com/grafana/grafana/releases
- The Launch of Grafana v5
- Grafana v5.0 Alpha Preview 2:32
- What’s New In Grafana v5.0 1:32

Install using Homebrew:

brew install grafana

The response at time of writing:

Downloading https://homebrew.bintray.com/bottles/grafana-6.6.1.mojave.bottle.tar.gz
==>
🍺  /usr/local/Cellar/grafana/6.6.1: 4,417 files, 189.6MB

For a list of commands:
```
grafana-cli
```
Server
Confirm server version installed:
```
grafana-server -v
```
The response at time of writing:
```
Version 9.1.6 (commit: unknown-dev, branch: main)
```
Configure Server

PROTIP: Avoid altering the defaults.ini file at: /opt/homebrew/Cellar/grafana/9.1.6/share/grafana/conf/defaults.ini
Copy file grafana.ini to make a custom.ini file in folder conf:
- /opt/homebrew/Cellar/grafana/9.1.6/.bottle/etc/grafana/grafana.ini
- /usr/local/etc/grafana/grafana.ini
Change only what needs customization by uncommentine lines, removing the semicolon from the beginning of each line.

Starting Server
To have launchd start grafana now and restart automatically at login:
```
sudo brew services start grafana
```
To stop:
```
sudo brew services stop grafana
```
Load the server:
```
brew tap homebrew/services
```
If you don’t want/need a background service, On an ARM M1 Mac, build the execution command with the version number at:
```
/opt/homebrew/Cellar/grafana/9.1.6/bin/grafana-server
```
Alternately, on a x86 Intel mac:
```
/usr/local/Cellar/grafana/9.1.6/bin/grafana-server
```

Define configuration files Alternately, you can just run on Linux:

sudo grafana-server --config=/usr/local/etc/grafana/grafana.ini \
--homepath /usr/local/share/grafana \
--packaging=brew cfg:default.paths.logs=/usr/local/var/log/grafana \
cfg:default.paths.data=/usr/local/var/lib/grafana \
cfg:default.paths.plugins=/usr/local/var/lib/grafana/plugins

UI

http://localhost:3000/login

Type in your password and click Allow.

grafana-accept-790x304

The response, for example:

INFO[03-11|17:19:43] Starting Grafana                         logger=server version=6.6.1 commit=unknown-dev branch=master compiled=2020-02-06T10:22:13-0700
INFO[03-11|17:19:43] Config loaded from                       logger=settings file=/usr/local/share/grafana/conf/defaults.ini
INFO[03-11|17:19:43] Config loaded from                       logger=settings file=/usr/local/etc/grafana/grafana.ini
INFO[03-11|17:19:43] Config overridden from command line      logger=settings arg="default.paths.data=/usr/local/var/lib/grafana"
INFO[03-11|17:19:43] Config overridden from command line      logger=settings arg="default.paths.logs=/usr/local/var/log/grafana"
INFO[03-11|17:19:43] Config overridden from command line      logger=settings arg="default.paths.plugins=/usr/local/var/lib/grafana/plugins"
INFO[03-11|17:19:43] Path Home                                logger=settings path=/usr/local/share/grafana
INFO[03-11|17:19:43] Path Data                                logger=settings path=/usr/local/var/lib/grafana
INFO[03-11|17:19:43] Path Logs                                logger=settings path=/usr/local/var/log/grafana
INFO[03-11|17:19:43] Path Plugins                             logger=settings path=/usr/local/var/lib/grafana/plugins
INFO[03-11|17:19:43] Path Provisioning                        logger=settings path=/usr/local/share/grafana/conf/provisioning
INFO[03-11|17:19:43] App mode production                      logger=settings
INFO[03-11|17:19:43] Initializing SqlStore                    logger=server
INFO[03-11|17:19:43] Connecting to DB                         logger=sqlstore dbtype=sqlite3
INFO[03-11|17:19:43] Starting DB migration                    logger=migrator
INFO[03-11|17:19:43] Initializing HTTPServer                  logger=server
INFO[03-11|17:19:43] Initializing BackendPluginManager        logger=server
INFO[03-11|17:19:43] Initializing PluginManager               logger=server
INFO[03-11|17:19:43] Starting plugin search                   logger=plugins
INFO[03-11|17:19:43] Initializing HooksService                logger=server
INFO[03-11|17:19:43] Initializing OSSLicensingService         logger=server
INFO[03-11|17:19:43] Initializing InternalMetricsService      logger=server
INFO[03-11|17:19:43] Initializing RemoteCache                 logger=server
INFO[03-11|17:19:43] Initializing QuotaService                logger=server
INFO[03-11|17:19:43] Initializing ServerLockService           logger=server
INFO[03-11|17:19:43] Initializing UserAuthTokenService        logger=server
INFO[03-11|17:19:43] Initializing RenderingService            logger=server
INFO[03-11|17:19:43] Initializing AlertEngine                 logger=server
INFO[03-11|17:19:43] Initializing DatasourceCacheService      logger=server
INFO[03-11|17:19:43] Initializing LoginService                logger=server
INFO[03-11|17:19:43] Initializing SearchService               logger=server
INFO[03-11|17:19:43] Initializing TracingService              logger=server
INFO[03-11|17:19:43] Initializing UsageStatsService           logger=server
INFO[03-11|17:19:43] Initializing CleanUpService              logger=server
INFO[03-11|17:19:43] Initializing NotificationService         logger=server
INFO[03-11|17:19:43] Initializing provisioningServiceImpl     logger=server
INFO[03-11|17:19:43] Backend rendering via phantomJS          logger=rendering renderer=phantomJS
WARN[03-11|17:19:43] phantomJS is deprecated and will be removed in a future release. You should consider migrating from phantomJS to grafana-image-renderer plugin. logger=rendering renderer=phantomJS
INFO[03-11|17:19:43] Initializing Stream Manager 
INFO[03-11|17:19:43] HTTP Server Listen                       logger=http.server address=[::]:3000 protocol=http subUrl= socket=

List the process

Upgrade Grafana
To upgrade, brew reinstall grafana

Data sources

Grafana has a growing list of data sources it can read from, via plug-ins:

CloudWatch
Elasticsearch (document database)
Graphite
Microsoft SQL Server
MySQL
ProtgreSQL
OpenTSDB (time series database)
InfluxDB
Prometheus

2h 43m PromQL for Prometheus 2.3.1 within a Docker image. He recommends having each project deploy its own Prometheus and Grafana instance rather than a central monitoring stack, sharing the admin burden. need to be highly available.

Datasource Plugin Thrunk (by Sven Nierlein)?

Filters

PROTIP: Define your filter statements before jumping into the Grafana UI.

Panels within Dashboard

Within each dashboard are panels with rows.

To add an additional panel within an established dashboard:

In your profile, get Editor Role.
No need to designate a data source.
Star the dashboard (Mark as favorite) on the top menu.
Set Home Dashboard in Preferences.
Add panel.
Choose visualization.
Graph.

Click on the share icon at the top menu.
Click Export tab.
Consider “Export for sharing externally”.
Click “Save to file”. Adjust folder and file name.

PROTIP: Save the file in GitHub for Infrastructure as Code. Then you won’t need to add a date/time stamp to the file.
Add tab, Add Row
Click on Row gear icon for options.
Update.
Click icon at top. Dummy graph appears.
Change Panel Title.
Select Data Source: “prometheus”.
Add query string
Set Panel Title.

Videos from Grafana.com

Torkel himself presents 12 videos in this YouTube Playlist also listed in the Grafana website:

1/. Building Graphite Queries Sep 15, 2014 [14:02]

A leaf note is a variable that contains data (such as count, dollars, etc.).

{} braces glob individual metrics together.

Templated Graphite Queries Sep 16, 2014 12:55
What’s New in Grafana 2.0 Apr 19, 2015 [13:48]
Installation & Configuration on Ubuntu / Debian 14:36
Installation & Configuration on Redhat / Centos 14:04
Adding data sources, users & organizations 15:17
Beginners guide to building dashboards 10:07
What’s New In Grafana 3.0 19:13
Elasticsearch 24:42
Dashboard Discovery & Sharing 7:40

Templates

Templates enable you to add variables

In Dashboard settings, Features tab, enable Templating checkbox. Close.
Click the gear next to VARIABLES.
Add a variable with a query type.
All option to use a * wildcard in the query.
Click the green > arrow to see values returned.
Also, add an interval type of variable.
Edit the time periods suggested in the Value field.
In the Graph, change the summarize to specify the variable.
VARIABLES: $internval auto [11:10]
Variable “smoothing” with Type custom [11:24]
Change the movingAverage value to $smoothing

Templating & Annotations GrafanaCon Mar 12, 2019 [24:47] by Daniel Lee

SQL

Grafana adds WHERE clause for time, such as $__timeFilter(time).

Plug-ins

grafana-cli plugins install sni-thruk-datasource

Videos

Monitoring Containerized Application Health with Docker
by Elton Stoneman 8 Aug 2018

How To Setup A Grafana Dashboard Step By Step - SQL4automation 21:00

Grafana Project Update & What’s New in v6.0 Mar 11, 2019 [40:31] Grafana Loki for Logs [11:40] In 2013 Grafana was re-written from AngularJs to React.

Setting up Prometheus and Grafana for monitoring your servers Jan 22, 2019 by Eddie Zaneski

How To Setup A Grafana Dashboard Step By Step Nov 5, 2019

Business Dashboards with Grafana and MySQL

Grafana at CERN by Borja Garrido Bear

What we Learned Integrating Grafana with Prometheus

Grafana Loki: Like Prometheus, But for logs by Tom Wilkie (@tom_wilkie), Grafana Labs CNCF [Cloud Native Computing Foundation]

Using Prometheus and Grafana for Monitoring my Power Usage Grafana

Using Postgres, Prometheus and Grafana for Storing, Analyzing and Visualizing Metrics by Erik Nordstrom from TimescaleDB

What we Learned Integrating Grafana with Prometheus Mar 1, 2018 by Dimitri Vanoverbeke from Percona

What Does Kubernetes Look Like? Performance Monitoring and Visualization with Grafana by Jacob Lisi

Automating Grafana Dashboards with Jsonnet Grafana

Weather, Power, and Market Forecasts with Grafana by Max Von Roden & Steffen Knott Mar 23, 2018

Workshop: Writing React Plugins by Terkel

Workshop: Grafana Plugin Showcase, Part 1 Workshop: Grafana Plugin Showcase, Part 2 Grafana

Workshop: Become a contributor - Get started developing Grafana Grafana

As of Feb. 14, 2020:

Gitter chat
2,522 follow @grafana on Twitter
243,522 follower it on LinkedIn
137,569 people follow its Facebook
Get release notices via User group is on Google Groups at
https://groups.google.com/forum/#!forum/go-cd
Subscribe to “The Pipeline” bi-monthly newsletter with continuous delivery news and events.

References about Grafana

https://www.youtube.com/watch?v=sTP7yzXmdFk&list=PLyJqGMYm0vnO9osZ-EBV6iu2l10muE2A- Free Grafana Complete course for beginners by I.T Security Labs

https://thenewstack.io/why-you-want-easy-to-setup-grafana-dashboards/

https://www.youtube.com/watch?v=Xj-TwAZtWHQ ELI5 - What I Learned Teaching Grafana to My Kids Grafana

https://www.youtube.com/watch?v=4qpI4T6_bUw How To Setup A Grafana Dashboard Step By Step SQL4automation

https://www.youtube.com/watch?v=oPumWaoNw5s&list=RDLVoPumWaoNw5s&start_radio=1&rv=oPumWaoNw5s Playlist: Guide to Grafana 101: Getting Started With (Awesome) Visualizations by Grafana Dev Evangelist Avthar Sewrathan

https://www.youtube.com/watch?v=w-c3KYKQQfs Grafana Tutorial For Beginners | Continuous Monitoring With Grafana | DevOps Training by edureka!

More on DevOps

This is one of a series on DevOps:

Wilson Mar

Grafana (visualization)

Install

Enterprise edition

Raspberry Pi

Releases

Server

Configure Server

Starting Server

UI

Upgrade Grafana