System monitoring is a process of checking the performance, health, capacity, and availability of a computer system. With the expansion of an organization, all its computing resources – the servers, devices, and services also increase.
If your system does not interact properly with the data sources, you will fail to deliver accurate results. Hence, proper monitoring tools are crucial for any system to work efficiently. Without them, it would be hard to identify and troubleshoot performance issues. They help ensure that the service is up and running smoothly.
Additionally, it will provide you with real-time analytics and a clear picture of how each application and infrastructure works. As a result, you will resolve issues before incidents occur and save your organization valuable time and money.
Grafana is a popular open-source solution used by professionals for data analytics and collecting metrics from large data to monitor apps, manage web app logs, and improve system performance. Like other tools, the solution is not limited to only offering data visualizations, instead, it helps track application behavior and aids businesses in understanding collected metrics.
Let us learn more about Grafana – its features, benefits, pricing, modules, and use cases. We will also discuss the tools that work great with Grafana.
What is Grafana?
Written in the Go programming language, Grafana is an open-source, powerful, and modern dashboard tool used by professionals for visualizing time series data, logs, and data collected from multiple applications. Most developers, data analysts, and ops teams who need to monitor the performance of applications, systems, and big data clusters in real time prefer Grafana over other tools.
Users can deploy it as a standalone application or as a plugin to other applications such as Elastic, (formerly known as Elasticsearch), or Apache Kafka.
It is a complete observability stack that allows users to create graphs and other visualizations of data collected from Prometheus, MySQL, PostgreSQL, and other databases. Using this powerful tool, any organization can pull out deep insights from the different databases. All you need is a dashboard to interact with Grafana and track the entire stack.
It also supports an alert feature that guides organizations about the issues or problems as they occur. It helps organizations receive notifications on time via email, webhooks, and Slack.
It is one of the best software available in the market that aids in visualizing and exploring data at scale. Using this software, users can turn all their time-series databases into simple graphs that give a clear picture of how the system and applications are performing.
Why use Grafana?
If you want to keep track of your infrastructure and logs, it is best to invest in software like Grafana. Grafana has a dashboard that helps visualize collected data and keeps proper track of users and events. Analysts use this information to improve operational efficiency and better decision-making.
Another reason why most companies choose Grafana is that it is a flexible system with a free-to-customize dashboard. Users can easily drag and drop, and resize the different panels per their choice to create a better visualization that meets their requirements. Also, it has a large community of users and contributors.
Grafana is open-source software with a plethora of visualization options and is easy to install on any platform. It helps define thresholds visually and notifies on time on discovering any incident.
If you are looking for a flexible and effective solution for your company that gives a better interpretation of complex or large data and monitors each event concerning the infrastructure, we recommend going for Grafana. You can also integrate it with other databases, including PostgreSQL, Influx DB, etc.
Grafana Modules
Grafana is a popular monitoring software but also comprises complimentary tools that help expand its functionality and turn the interface into a monitoring system. These modules are:
- Graphite A monitoring tool that collects, stores, and visualizes time-series data for query and analysis.
- Carbon–relay–ng Its role is to collect and transfer data from multiple sources into Graphite.
- Cortex It collects and stores data in Prometheus format for a long duration. It later uses the stored information for long-term capacity planning and historical data analysis.
- Loki A log aggregation system designed for storing and labeling log data of each application and infrastructure.
- Tanka A configuration utility best for Kubernetes cluster.
Grafana Pricing
Grafana is available in three packages – Community Edition, Grafana Enterprise Stack, and Grafana Cloud.
The Community Edition package is also referred to as the Grafana Open Source package. It is a free edition without any professional support. The edition is compatible with Windows, Docker, Linux, and macOS platforms. Grafana also has a large community forum for users to ask and participate in discussions.
The Grafana Enterprise Stack package, on the other hand, is a paid version with professional support services included in the package. The Enterprise package comprises different modules:
Grafana Enterprise
The community edition features along with professional support services are available in the package. Also, the proprietary plugins responsible for automatically feeding data in from third-party tools are included.
- Grafana Enterprise Logs It is a paid version that includes a combination of Loki and Grafana tools, access controls, and professional support services.
- Grafana Enterprise Metrics Another scalable, paid package provides centralized access control, authentication features, self-monitoring options, and a global view of metrics. Apart from this, it allows users to quickly deploy and customize their approach per the need.
- Grafana Enterprise Traces Your organization gets access to scale from one to hundreds of tenants. Additionally, your team members can aggregate data from multiple clusters and create tokens. Based on the open-source project Tempo, the distributed tracing system helps feed in Prometheus data and maintain logs from Loki.
The third edition or package, Grafana Cloud is a SaaS plan that offers full-stack monitoring and alert options. The users can collect, store, and visualize their data. It also covers modern distributed systems techniques and 24×7 on-call support. You get access to 30 days of log and trace retention and automatic upgrades under the package.
For the free tier, only three active users can access 10,000 metrics, 50GB logs, and 50GB trace data.
Features of Grafana
Let us have a look at some of the best features of the Grafana monitoring tool:
- Dashboard Templating It has a customizable dashboard used that helps visualize data in graphical form, keeps track of each event, and allows users to share data among team members.
- Authentication Grafana supports LDAP and OAuth authentications, i.e., only authorized users with correct usernames and passwords can access the data, and share, or grant access to other team members in the internal systems.
- Provisioning The provisioning feature allows users to script anything and manage several dashboards at the same time. Users can also create a single dashboard and script it accordingly to generate multiple similar dashboards.
- SQL data sources Users have the access to turn their SQL databases into metric data using this feature.
- Annotations Grafana users can generate annotations or mark graphs to control data manually or fetch it from an external source. Annotations act like a piece of additional information that helps correlate data on discovering an issue.
- Permissions Using the Permission feature, you can control how much data you want to make visible to team members or share with admins. Even if you are sharing dashboards, you can set permissions for folders and manage data source levels.
- Kiosk mode and playlists If you enable the view-only kiosk mode, all the user interface elements will be hidden. You can also create a playlist of your Grafana dashboards per your choice.
- Alert Option On noticing any suspicious activity or issue, the tool immediately sends alerts and notifications to the admin using Slack or other communication options.
- Custom Plugins Grafana supports several plugins that help extends its functionality and improve operational efficiency. You can also visualize a timestamp using the plugins supported by the tool.
Advantages of Grafana
- Users can create customized multiple dashboards via scripts
- Multi-tenanted options are available for managed service providers
- Using its single console, users can keep track of multiple sites
- Users can integrate the Grafana monitoring tool with Elastic, InfluxDB, Prometheus, MySQL, Splunk, and other databases
- Notifies admin instantly on discovering issues using email, PagerDuty, or Slack
- Grafana can be easily configured and supports a set of customizable visualization panel options
- Offers charts, histograms, heatmaps, and other options that give a clear picture of the collected data
- Grafana is an easy-to-use and flexible monitoring tool
- Supports a wide range of plugins that help extend the tool’s functionalities.
- Hosted by on-premises or cloud platforms
Grafana Use Cases
At present, Grafana has crossed 950,000 active installations and is used by various organizations for various purposes. Here are a few use cases of Grafana:
Optum uses Grafana for viewing healthcare websites
Optum’s monitoring team uses the Grafana Enterprise package and a number of plugins, including Splunk, Datadog, New Relic, etc., to get an accurate idea of the user experience and the infrastructure performance. By delivering a better user experience, the portal is in a way improving Net Promoter Scores (NPS) or financial performance.
Cisco uses Grafana for Better Visibility
Cisco Live is the biggest event that covers certification testing, product announcements, showcases, and other training programs. Around 28,000 attendees from the U.S. and approximately 17,000 attendees from Europe come together for the event. The program requires 650 network switches, a mobile containerized data center, 2,300 wireless access points, and dual 100 Gigabit per second Internet links to smoothly run the event.
To get an accurate figure of the network statistics and showcase interesting metrics about the event, the Network Operations Center team uses the Grafana monitoring tool. It also updates users about consumption, performance, availability, and other criteria.
Tools that work with Grafana
Here are the following integrations supported by Grafana Monitoring Tool:
- MySQL An open-source RDBMS used by organizations to store, manage, and modify data. MySQL is generally used for large and complicated analytical processes. Once MySQL is integrated with Grafana, admins gain access to send metrics and logs automatically from the database to Grafana Cloud. It also includes alert options and prebuilt dashboards that aid organizations in monitoring relevant metrics.
- Elastic A search engine with an HTTP web interface that also supports integration with the Grafana monitoring tool. Once integrated, it activates the agent who sends metrics to Grafana Cloud with the default dashboard for visualization.
- Docker A popular platform that separates the application from the infrastructure and aids in building, shipping, and running applications. The Docker integration monitors and displays the container utilization. It allows admins to install pre-built dashboards and alerts that aid in monitoring logs for Docker containers.
- Jenkins An open-source automation server used by developers for testing, deploying, building, and monitoring software projects. If your organization wants to accelerate the software development process and deliver accurate results, install Jenkins. You can also run the software in servlet containers, for example, Apache Tomcat. Most companies integrate Jenkins with Grafana Cloud to learn about the state of the build infrastructure and the duration of defined jobs. It also enables the admin to monitor cluster health and visualize them.
- MongoDB A distributed database used for the cloud. Modern application developers integrate MongoDB with Grafana with the help of Percona MongoDB Exporter. It is essential to check that each node of the database is connected to the agent instance as the embedded Grafana agent or exporter does not collect metrics from a host of sources. It only helps monitor a MongoDB Cluster and sends timely alerts.
Install and Configure Grafana
In this section, we will install Grafana and then integrate the MySQL server to visualize MySQL data via the Grafana dashboard:
Install Grafana
By default, the Grafana package is not included in the Ubuntu default repository. So you will need to add the Grafana repository to APT.
First, you will need to install the required dependencies on your server. You can install all of them using the following command:
apt-get install gnupg2 curl wget git software-properties-common -y
Once all the dependencies are installed, add the Grafana key and repository with the following command:
curl https://packages.grafana.com/gpg.key | apt-key add - add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
Once the repository is added, update the repository and install Grafana with the following command:
apt-get update -y apt-get install grafana -y
Once Grafana is installed, start the Grafana service using the following command:
systemctl start grafana-server
You can now check the status of Grafana with the following command:
systemctl status grafana-server
You should get the following output:
● grafana-server.service - Grafana instance
Loaded: loaded (/lib/systemd/system/grafana-server.service; disabled; vendor preset: enabled)
Active: active (running) since Sun 2022-08-28 08:16:37 UTC; 5s ago
Docs: http://docs.grafana.org
Main PID: 3833 (grafana-server)
Tasks: 10 (limit: 2347)
Memory: 36.6M
CGroup: /system.slice/grafana-server.service
└─3833 /usr/sbin/grafana-server --config=/etc/grafana/grafana.ini --pidfile=/run/grafana/grafana-server.pid --packaging=deb cfg:>
Aug 28 08:16:40 grafana grafana-server[3833]: logger=live.push_http t=2022-08-28T08:16:40.190299936Z level=info msg="Live Push Gateway initia>
Aug 28 08:16:40 grafana grafana-server[3833]: logger=ticker t=2022-08-28T08:16:40.219427742Z level=info msg=starting first_tick=2022-08-28T08>
Aug 28 08:16:40 grafana grafana-server[3833]: logger=infra.usagestats.collector t=2022-08-28T08:16:40.313263194Z level=info msg="registering >
Aug 28 08:16:40 grafana grafana-server[3833]: logger=server t=2022-08-28T08:16:40.313831461Z level=info msg="Writing PID file" path=/run/graf>
Aug 28 08:16:40 grafana grafana-server[3833]: logger=provisioning.alerting t=2022-08-28T08:16:40.334874257Z level=info msg="starting to provi>
Aug 28 08:16:40 grafana grafana-server[3833]: logger=provisioning.alerting t=2022-08-28T08:16:40.335341787Z level=info msg="finished to provi>
Aug 28 08:16:40 grafana grafana-server[3833]: logger=http.server t=2022-08-28T08:16:40.345312357Z level=info msg="HTTP Server Listen" address>
Aug 28 08:16:40 grafana grafana-server[3833]: logger=ngalert t=2022-08-28T08:16:40.345735931Z level=info msg="warming cache for startup"
Aug 28 08:16:40 grafana grafana-server[3833]: logger=grafanaStorageLogger t=2022-08-28T08:16:40.37754408Z level=info msg="storage starting"
Aug 28 08:16:40 grafana grafana-server[3833]: logger=ngalert.multiorg.alertmanager t=2022-08-28T08:16:40.384317318Z level=info msg="starting >
By default, The Grafana listens on port 3000. You can check it with the following command:
ss -antpl | grep grafana
You should see the following output:
LISTEN 0 4096 *:3000 *:* users:(("grafana-server",pid=3833,fd=8))
Create Database and Tables in MariaDB
First, install the MariaDB database with the following command:
apt install mariadb-server -y
Next, log in to the MariaDB shell with the following command:
mysql -u root -p
Once you are logged in, create a database and user with the following command:
CREATE USER 'mysqluser'@'localhost' IDENTIFIED BY 'password'; CREATE DATABASE mysqldb;
Next, grant all the privileges to the MariaDB database with the following command:
GRANT ALL PRIVILEGES ON mysqldb.* TO 'mysqluser'@'localhost';
Next, flush the privileges with the following command:
FLUSH PRIVILEGES;
Next, switch the database to the mysqldb and create a table with the following command:
use mysqldb; CREATE TABLE ScheduleDetails( TrainId INT NOT NULL, StationName VARCHAR(20) NOT NULL, TrainName VARCHAR(20) NOT NULL, ScheduledlArrivalTime TIME NOT NULL, PRIMARY KEY(TrainId ) );
Next, insert some data into the table with the following command:
INSERT INTO ScheduleDetails (TrainId, StationName, TrainName, ScheduledlArrivalTime ) VALUES (12345, 'NJP', 'Saraighat Express', "17:04:22");
Next, verify the inserted data using the following command:
SELECT *, ADDTIME(ScheduledlArrivalTime, "00:10:00") AS ExpectedArrivalTime FROM ScheduleDetails;
You will get the following output:
+---------+-------------+-------------------+-----------------------+---------------------+ | TrainId | StationName | TrainName | ScheduledlArrivalTime | ExpectedArrivalTime | +---------+-------------+-------------------+-----------------------+---------------------+ | 12345 | NJP | Saraighat Express | 17:04:22 | 17:14:22 | +---------+-------------+-------------------+-----------------------+---------------------+
Integrate Grafana with MySQL
Now, open your web browser and access the Grafana interface using the URL http://your-server-ip:3000. You should see the Grafana login page:
Provide default username and password as admin/admin then click on the Login button. You should see the password change screen:
Change your default admin password and click on the Submit button. You should see the Grafana dashboard on the following page:
Click on Add your first data source. You should see the following page:
Search for MySQL and click on it, You should see the MySQL connection page:
Provide your MySQL host, username, and password then click on the Save & test button.
Go back to the Grafana dashboard and click on Create your first dashboard. You should see the following page:
Click on the Add a new panel. You should see the following page:
Provide the title and description of the Dashboard, select MySQL as a data source, type your MySQL query and click on the Apply button. You should see your MySQL data on the following page:
Conclusion
For smooth operations and efficient working of systems, it is essential for organizations to invest in monitoring tools that give a clear picture of how each application and infrastructure works. Grafana is a leading open-source solution that monitors large data sets and complex information to improve system performance.
It not only eases the process of collecting metrics and visualization but also helps track application behavior. Written in the Go programming language, most companies use it for visualizing time series data, logs, and other data collected from a host of sources. It also creates graphs for the data collected from databases, such as Splunk, MySQL, etc.
It is an easy-to-install open-source solution that helps define thresholds visually and notifies about incidents in real time. We have also discussed Grafana modules and how the complementary tools help expand Grafana’s functionality.
Check out the above-listed features, benefits, use cases, and pricing plans before coming to a conclusion. We have also shared a list of tools that integrate best with the Grafana Monitoring Tool, offer an accurate figure of the network statistics, and display interesting metrics.
