DevOpsil
Prometheus28 commands

Prometheus & PromQL Cheat Sheet

Essential PromQL queries and Prometheus commands for metrics, alerting, service discovery, and dashboarding.

Basic Queries

CommandDescription
up
Check if targets are up (1) or down (0)
node_cpu_seconds_total
Total CPU seconds by mode
http_requests_total{status="200"}
Filter by label value
http_requests_total{status=~"5.."}
Regex match (5xx errors)
http_requests_total{method!="GET"}
Negative match
metric_name[5m]
Range vector (last 5 minutes)

Aggregation & Math

CommandDescription
rate(http_requests_total[5m])
Per-second request rate
increase(http_requests_total[1h])
Total increase over 1 hour
sum(rate(http_requests_total[5m])) by (status)
Sum rate grouped by status
avg(node_cpu_seconds_total) by (instance)
Average by instance
topk(5, rate(http_requests_total[5m]))
Top 5 by request rate
count(up == 1)
Count healthy targets
histogram_quantile(0.99, rate(http_duration_seconds_bucket[5m]))
P99 latency

Common Patterns

CommandDescription
100 - (avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)
CPU usage %
(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes * 100
Memory usage %
(node_filesystem_size_bytes - node_filesystem_free_bytes) / node_filesystem_size_bytes * 100
Disk usage %
rate(http_requests_total{status=~"5.."}[5m]) / rate(http_requests_total[5m]) * 100
Error rate %
predict_linear(node_filesystem_free_bytes[1h], 4*3600) < 0
Disk full in 4 hours?

Alert Rules

CommandDescription
expr: up == 0 for: 5m
Alert when target down for 5m
expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.05
Alert on high error rate
expr: (node_filesystem_avail_bytes / node_filesystem_size_bytes) < 0.1
Disk space < 10%
labels: severity: critical
Set alert severity label
annotations: summary: "Instance {{ $labels.instance }} down"
Alert annotation template

promtool & Admin

CommandDescription
promtool check config prometheus.yml
Validate config file
promtool check rules alerts.yml
Validate alert rules
curl localhost:9090/-/reload -X POST
Hot-reload config
curl localhost:9090/api/v1/targets
List scrape targets
curl localhost:9090/api/v1/query?query=up
Query via HTTP API