Fix Grafana Dashboards That Load Forever

The Symptom: Dashboard Spinner That Never Stops

You navigate to a Grafana dashboard and see the loading spinner on every panel. Some panels eventually load after 30-60 seconds. Others time out entirely with:

Panel plugin error: Query timeout

The browser tab might even become unresponsive.

Root Cause

Slow dashboards are almost always caused by expensive queries overwhelming the backend datasource. Specific causes include:

• Queries scanning too many time series across a wide time range
• Too many panels (50+) all firing queries simultaneously on page load
• Missing recording rules forcing Prometheus to compute aggregations at query time
• Wide default time range (30 days) on a dashboard designed for hourly views
• No $__interval usage causing unnecessarily high-resolution data for large time windows

Step-by-Step Fix

1. Identify the Slow Queries

Open the dashboard, then press Ctrl+Shift+J (Chrome DevTools) and check the Network tab. Look for /api/ds/query requests that take the longest. Note which panel they belong to.

Alternatively, use Grafana's built-in query inspector:

1. Click the panel title
2. Select Inspect > Query
3. Check the query execution time shown in the response

2. Add $__rate_interval and Limit Series

Replace raw PromQL with template variables that scale with the time range:

# Bad: always computes 1-minute rate, expensive over 30 days
rate(http_requests_total[1m])

# Good: rate interval adjusts to the dashboard time range
rate(http_requests_total[$__rate_interval])

Limit the series returned:

# Bad: returns all endpoints
sum by (endpoint) (rate(http_requests_total[$__rate_interval]))

# Good: only top 10 endpoints
topk(10, sum by (endpoint) (rate(http_requests_total[$__rate_interval])))

3. Create Recording Rules for Heavy Aggregations

If a query aggregates across thousands of series, pre-compute it in Prometheus:

# prometheus-rules.yml
groups:
  - name: dashboard-optimizations
    interval: 1m
    rules:
      - record: job:http_requests:rate5m
        expr: sum by (job) (rate(http_requests_total[5m]))
      - record: job:http_request_duration:p99
        expr: histogram_quantile(0.99, sum by (job, le) (rate(http_request_duration_seconds_bucket[5m])))

Then reference job:http_requests:rate5m in your dashboard instead of the raw aggregation.

4. Reduce Panel Count and Use Lazy Loading

For dashboards with many panels, enable row-based loading:

1. Group panels into collapsible rows
2. Collapsed rows don't execute queries until expanded
3. Aim for no more than 15-20 visible panels at a time

5. Set Appropriate Time Ranges and Refresh

Set the dashboard default time range to something reasonable:

1. Click the gear icon (Dashboard settings)
2. Under Time Options, set the default time range to Last 6 hours instead of Last 30 days
3. Set Auto-refresh to 30s or 1m instead of 5s

6. Increase Datasource Timeout

If queries are legitimately slow but eventually complete:

Go to Connections > Data sources > Prometheus > Additional settings:

• Set Scrape interval to match your Prometheus config (e.g., 15s)
• Set Query timeout to 120s

7. Verify Improvement

Reload the dashboard. Open Chrome DevTools Network tab and confirm that query response times are under 5 seconds. Check that total page load is under 10 seconds.

Prevention Tips

• Enforce a panel limit per dashboard (20 panels max as a team guideline).
• Use recording rules for any aggregation queried by more than one dashboard.
• Add Min step to panels to prevent Grafana from requesting more data points than the panel width in pixels.
• Review dashboards quarterly. Remove panels nobody looks at.
• Use Grafana's usage insights (Enterprise) or query logs to find the most expensive queries.