DevOpsil
NginxHigh

Fix: Nginx 504 Gateway Timeout

Users see '504 Gateway Timeout' after a long wait

!Symptoms

  • Users see '504 Gateway Timeout' after a long wait
  • Nginx error log shows 'upstream timed out'
  • Long-running API requests fail consistently
  • Only affects certain endpoints, not the whole site

?Root Causes

  • Upstream application taking too long to respond (slow queries, heavy processing)
  • Nginx proxy_read_timeout too short for the operation
  • Database or external API the upstream depends on is slow
  • Deadlock or resource contention in the upstream application
  • Network latency between Nginx and the upstream server

#Diagnosis Steps

  1. 1Check Nginx error log for timeout messages: `tail -f /var/log/nginx/error.log`
  2. 2Identify which endpoint is slow from access logs: `awk '$9 == 504' access.log`
  3. 3Test the upstream directly to measure response time: `time curl http://upstream:port/endpoint`
  4. 4Check upstream application logs for slow operations
  5. 5Monitor database query times for the affected endpoints

>Fix

  1. 1Increase Nginx timeout values: proxy_read_timeout, proxy_connect_timeout, proxy_send_timeout
  2. 2Optimize the slow upstream endpoint (database indexes, caching, async processing)
  3. 3Move long-running tasks to a background job queue (Celery, Sidekiq, Bull)
  4. 4Add request buffering: `proxy_buffering on` with appropriate buffer sizes
  5. 5Implement pagination or streaming for large data responses

*Prevention

  • Set appropriate timeouts at every layer (Nginx, app, database)
  • Implement request timeouts in the application code itself
  • Use async processing for operations that take more than a few seconds
  • Monitor response time percentiles (p95, p99) and alert on degradation
  • Load test endpoints with realistic payloads before release

Related Error Messages

504 Gateway Timeoutupstream timed out (110: Connection timed out)upstream timed out (110: Operation timed out)proxy_read_timeout