Microservices are solving many architectural problems that have plagued developers and operations for years. But while the benefits are now clear, many professionals aren't fully aware of the many challenges that come with microservices. A significant portion of those center around monitoring and observability.
Microservices add complexity to your system, communication, and network, which makes monitoring more complicated. Thankfully, you can find lots of tutorials, books, tools, and case studies from organizations that have successfully implemented effective monitoring and observability in a microservices environment.
Here is our list of the most popular and useful tools and resources shared within the software engineering, site-reliability engineer (SRE), and DevOps communities.
Tutorials and guides for monitoring microservices
How Hootsuite Manages Its Growing Microservices Landscape
A software engineer from Hootsuite explains how service relationships, dependencies, and status are paramount to keeping his team, the build pipeline, and the application running smoothly. This talk includes a section on the API that's baked into Hootsuite's service-oriented architecture, enabling them to monitor and explore application service dependency graphs in real time.
Q&A James Turnbull: The Art of Monitoring in the Age of Microservices
James Turnbull is the author of The Art of Monitoring. This conversation with him provides a good overview of the state of microservices monitoring.
Monitoring Microservices Architectures: Enterprise Best Practices
This roundup talks to engineers from IBM, the Financial Times, and Netflix about the pain points of microservices monitoring, then moves on to describe each firm's best practices for maintaining and monitoring microservices.
Monitoring for Distributed and Microservices Deployments
This guide explains the challenges of building distributed architectures and the required changes to scale the monitoring of microservices. It emphasizes the ways in which various system architectures can influence how you respond to incidents and resolve them.
Monitoring a Microservices-Based Application
Despite the benefits of using a microservices-based architecture, they add a lot of "noise." Monitoring a microservices-based environment without taking this noise into account can be very confusing. Here's how you can eliminate at least some of it.
Future-proof Docker and Microservices with Monitoring
This post explains the relationship between containers (Docker containers in this example) and microservices. It also covers the advantages of using these technologies and the challenges of monitoring Docker. “What should you look for in a Docker monitoring solution?” Three things, the author says.
The Future of Microservices Monitoring & Instrumentation
This article explains how containerization and microservices changed the way developers write and run their applications. The author predicts what new monitoring and instrumentation practices will be utilized in 2018.
Monitoring as a Service: a Modular System for Microservice Architecture
This blog post offers up a real-world example of a monitoring architecture for microservices that uses a variety of open-source tools, including Graphana, Collectd, Heapster, and Brubeck.
Monitoring a Production-Ready Microservice
This excerpt from the book Production-Ready Microservices, by Susan Fowler, provides a list of useful questions to assess the production-readiness of microservices and a microservice ecosystem. The questions are related to key metrics, logging, dashboards, alerting, and on-call rotations to give you an idea of what kinds of tools and processes you need to have in place to effectively monitor microservices.
Communication Between Microservices: How to Avoid Common Problems
This pitfalls-style post explains the performance, fault tolerance, and monitoring challenges when running microservices. Then it gives the reader some tools and suggestions to overcome those challenges.
Node.js Performance Monitoring with Prometheus
This detailed guide on Node.js monitoring with the Prometheus monitoring framework will help you understand Node.js-based microservices monitoring by covering code instrumentation (and its risks), data collection, data visualization, queries, alerting, and Kubernetes integration.
Monitoring Envoy and Ambassador on Kubernetes with the Prometheus Operator
This tutorial shows you how to set up monitoring in a cloud-native microservices environment using Prometheus Operator to monitor Envoy proxy. As the author explains, one reason for Envoy’s growing popularity is its emphasis on observability. Instead of using Envoy directly, the tutorial uses Ambassador.
Distributed Tracing with Zipkin, Stackdriver Trace in Kubernetes
You can use another proxy, Zipkin, with Kubernetes (and Google Container Engine) to send trace information when monitoring microservices. Learn how it works in this article.
In the Land of Microservices, the Network is the King(maker)
Networking knowledge is critical in microservices monitoring. Here's why.
Planning Your Docker-based Microservice Stack
Here are several considerations for deploying a Docker-based microservices architecture—and some best practices for externalizing, monitoring, and managing microservices.
Monitoring & observability tools
A Curated List of Microservice Architecture Related Principles and Technologies
This article offers up a great list of tools for monitoring and debugging microservices.
Prometheus
Prometheus is an open-source service-monitoring system and time-series database.
Spigo
Spigo is a tool that simulates protocol interactions in the Go language. You can use it to run microservices architecture simulations and generate new visualizations and metrics.
Collectd
This daemon collects system and application performance metrics over time, and provides mechanisms to store the values in a variety of ways. It gathers metrics from many sources, including the operating system, applications, log files, and external devices. It can then store this information or make it available over the network.
Riemann
A tool for monitoring distributed systems, Riemann can forward any event stream to Graphite, a popular monitoring framework and time-series database. It also provides phone, SMS, and email alerts.
Heapster
Use Heapster to monitor Kubernetes container clusters. It collects and interprets various signals such as compute resource usage and lifecycle events, supporting multiple sources of data.
cAdvisor
This container-monitoring tool, developed by Google, natively supports Docker containers. The cAdvisor daemon, which collects, aggregates, processes, and exports information about containers, is available in a graphical version.
Sysdig
Sysdig is a monitoring and troubleshooting tool that captures system calls and events from the Linux kernel, and lets you to save, filter, and analyze the data from the command line or from within a desktop app. The open-source developers of Sysdig describe it as “strace + tcpdump + htop + iftop + lsof + wireshark for your entire system.”
Grafana
A metrics dashboard, Grafana integrates with many data sources, including Graphite, CloudWatch, Prometheus, Elsaticsearch, and InfluxDB.
Trace
The Trace visualized distributed tracing platform is designed for Node.js microservices.
Sensu
Sensu helps developers monitor servers, services, application health, and business KPIs. It collects and analyzes custom metrics and can send failure notifications. You can use Sensu with containers such as Docker, RKT, and LXC.
Are there any other resources that you find particularly useful in the field of microservices monitoring and observability? Share them in the comments below.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.