Why Container Monitoring Matters in Production

Docker containers have transformed how teams deploy and scale applications by packaging code with its dependencies into lightweight, portable units. However, the density of containers on a single host—often dozens or even hundreds—creates a blind spot: without insight into resource consumption, a single runaway container can starve others of CPU, memory, or disk I/O, leading to cascading failures. cAdvisor, short for Container Advisor, was created by Google to fill this gap. It provides per-container resource metrics in real time, helping operators understand exactly what each container is using and when intervention is needed.

In this guide, we’ll walk through everything you need to know about monitoring Docker container resource usage with cAdvisor, from a basic single-host setup to integrating it into a comprehensive observability stack with Prometheus and Grafana. Whether you run a few containers locally or manage a fleet of production servers, cAdvisor offers the visibility required to keep your infrastructure healthy.

What is cAdvisor?

cAdvisor is an open-source agent that runs as a DaemonSet (or standalone container) and collects resource usage and performance characteristics of running containers. It was open-sourced by Google and is widely adopted because it requires minimal configuration, exposes metrics via a REST API, and includes a built-in web dashboard for quick visual inspection.

Key capabilities include:

  • Automatic discovery of all containers on a host (including Kubernetes pods when deployed as a DaemonSet).
  • Rich metric collection covering CPU, memory, network, filesystem, and storage I/O.
  • Historical data retention (configurable) for trend analysis.
  • Export endpoints for Prometheus, InfluxDB, and other time-series databases.
  • Integration with cgroups to enforce resource limits and reservations.

Because cAdvisor itself runs as a container, it’s trivial to deploy anywhere Docker runs—making it the de facto standard for single-host monitoring.

Deploying cAdvisor with Docker

The simplest way to start monitoring your containers is to run cAdvisor as a Docker container. The following command mounts the necessary host volumes so cAdvisor can access the Docker socket, the kernel’s cgroup hierarchy, and the Docker storage directories:

docker run -d \
  --name=cadvisor \
  --restart=always \
  -p 8080:8080 \
  --volume=/var/run/docker.sock:/var/run/docker.sock:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  gcr.io/cadvisor/cadvisor:latest

After the container starts, open http://localhost:8080 to see the live dashboard. The overview page lists all running containers along with aggregated metrics. Clicking any container name drills into detailed graphs and tables for CPU, memory, network, and disk.

For production deployments, you should also consider:

  • Resource constraints on the cAdvisor container itself (e.g., 0.5 CPU, 256 MB memory) to prevent it from interfering with workload containers.
  • Persistent storage for the metrics database if you want historic data beyond the default retention (usually a few minutes).
  • Securing the dashboard behind a reverse proxy with authentication, since cAdvisor exposes detailed runtime information.

Using Docker Compose for cAdvisor

When you have a multi-container application, Docker Compose simplifies deployment. Here’s a sample docker-compose.yml that runs cAdvisor alongside a simple Nginx container:

version: '3.8'
services:
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    restart: always
    ports:
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
  nginx:
    image: nginx:latest
    container_name: nginx-demo
    ports:
      - "80:80"

Run docker-compose up -d and cAdvisor will automatically detect the Nginx container and start collecting its metrics.

Understanding the Key Metrics

cAdvisor exposes dozens of metrics per container. The most important ones for day-to-day monitoring are:

CPU Usage

CPU metrics include instantaneous usage in cores (e.g., 0.75 cores), cumulative usage in nanoseconds, and the CPU throttling count. Monitoring CPU helps identify containers that are over-consuming or being throttled due to limits. The dashboard shows a time-series graph so you can correlate spikes with application events.

Memory Usage

Memory metrics include working set size, page faults, and limits. The working set is the most reliable indicator of actual memory demand. If a container’s memory usage approaches its limit (--memory flag), cAdvisor will show a warning. This is critical for preventing out-of-memory (OOM) kills.

Network Traffic

cAdvisor tracks bytes and packets sent/received per network interface. This helps you assess network IO bottlenecks, especially for data-intensive services like reverse proxies or databases. The dashboard breaks down traffic per container with separate read/write lines.

Disk I/O and Filesystem Usage

Storage metrics include read/write operations and latency, as well as filesystem capacity. If a container writes logs or temporary files aggressively, you can see the I/O impact immediately. The filesystem usage graph also shows how much of the allocated storage is consumed.

Accelerator (GPU) Metrics

For machine learning workloads, cAdvisor can collect NVIDIA GPU utilization and memory usage if the NVIDIA Container Toolkit is installed. This is a powerful feature for AI/ML engineers managing GPU clusters.

Integrating cAdvisor with Prometheus and Grafana

While the cAdvisor dashboard is great for quick checks, production systems need long-term storage and flexible alerting. The standard solution is Prometheus for metric collection and Grafana for visualization.

Exposing Metrics to Prometheus

cAdvisor exposes Prometheus-compatible metrics at /metrics. Add a Prometheus scrape target in your prometheus.yml:

scrape_configs:
  - job_name: 'cadvisor'
    scrape_interval: 15s
    static_configs:
      - targets: ['localhost:8080']

On each host where cAdvisor runs, Prometheus will collect container-level metrics with labels for container name, image, and pod (if using Kubernetes). The metric names all start with container_ (e.g., container_cpu_usage_seconds_total).

Building Dashboards in Grafana

To visualize cAdvisor data in Grafana, you can use one of the many pre-built dashboards available on the Grafana community library. Alternatively, create custom panels:

  • CPU usage per container: rate(container_cpu_usage_seconds_total{container_label_com_docker_compose_service!=""}[5m])
  • Memory working set: container_memory_working_set_bytes{name!=""}
  • Network received bytes: rate(container_network_receive_bytes_total[5m])
  • Filesystem usage: container_fs_usage_bytes

With Grafana, you can set thresholds, correlate metrics from multiple hosts, and share dashboards with your team.

Best Practices for Using cAdvisor

To get the most out of cAdvisor without overcomplicating your setup, follow these guidelines:

  • Run one cAdvisor per host. Do not attempt to monitor remote Docker daemons from a single instance; mount the local Docker socket for accurate data.
  • Limit historical data retention. cAdvisor defaults to keeping ~2 minutes of historical data in memory. If you need longer retention, forward metrics to Prometheus or InfluxDB and set a retention window there.
  • Secure the /metrics endpoint. If you expose cAdvisor’s port beyond localhost, authenticate requests. The /metrics endpoint can reveal sensitive information about your container environment.
  • Monitor cAdvisor itself. Add a health check and resource limits to the cAdvisor container. If cAdvisor stops, you lose visibility into the host’s containers.
  • Use labels and filters. When querying Prometheus, filter by container labels (like container_label_com_docker_compose_service) to avoid including cAdvisor’s own metrics in your dashboards.

Use Cases & Scenarios

cAdvisor shines in several real-world situations:

  • Capacity planning: By collecting CPU and memory trends over weeks, you can right-size container limits before moving to production.
  • Debugging resource contention: When a host’s CPU spikes, the cAdvisor dashboard instantly shows which container(s) are responsible.
  • Billing/showback: For shared infrastructure, cAdvisor metrics provide per-tenant resource consumption data for internal chargebacks.
  • Kubernetes clusters: cAdvisor is already embedded in the kubelet, but running it standalone gives you an independent view without relying on the Kubernetes API.

Comparison with Other Monitoring Tools

While cAdvisor is excellent for per-container metrics, it’s not the only option:

  • Docker stats: The built-in docker stats command provides a live stream but lacks history, API access, and the ability to export to time-series databases.
  • Prometheus node_exporter: Provides host-level metrics but does not break down by container.
  • Datadog Agent / New Relic: These commercial solutions offer container monitoring with pre-built dashboards and alerting, but at a cost per host.
  • Netdata: Another open-source option with a rich UI, but heavier than cAdvisor and more focused on host metrics.

cAdvisor strikes a balance between simplicity, zero cost, and deep container-level metrics. It’s the ideal choice for teams that already use Prometheus and want a lightweight, battle-tested solution.

Potential Pitfalls and How to Avoid Them

Even a straightforward tool like cAdvisor has edge cases:

  • Running on non-Docker runtimes: cAdvisor only supports Docker rootful and rootless modes. For containerd or CRI-O, consider using cAdvisor’s standalone mode with the --docker_only flag set to false.
  • High cardinality: If containers are created and destroyed rapidly (e.g., short-lived batch jobs), the number of unique label combinations can overwhelm Prometheus. Use Prometheus’s metric_relabel_configs to drop unnecessary labels.
  • Outdated image: cAdvisor is rarely updated, but image tags like latest still receive security patches. Pin to a specific version and update periodically.

Conclusion

Monitoring Docker container resource usage with cAdvisor is one of the fastest ways to gain observability into your containerized infrastructure. Its lightweight design, automatic detection, and seamless integration with Prometheus make it an indispensable tool for developers and operators alike. By following the deployment and best practices outlined in this guide, you can ensure that your containers run within their limits, your hosts remain stable, and your applications stay performant.

For further reading, check out the official cAdvisor GitHub repository (github.com/google/cadvisor), the Docker documentation on resource constraints (docs.docker.com), and the Prometheus integration guide (prometheus.io).