Fix AWS EC2 Instance 'Status Check Failed' Errors

The Error: Status Check Failed

In the EC2 console, your instance shows a red "2/2 checks failed" or "1/2 checks failed" status. The instance is unreachable via SSH, and your application is down.

AWS has two types of status checks:

• System Status Check -- problems with the underlying AWS hardware
• Instance Status Check -- problems with the OS or instance configuration

Root Cause

System status check failures are caused by:

• AWS hardware issues on the host machine
• Loss of network connectivity to the physical host
• Power issues on the host

Instance status check failures are caused by:

• Corrupted file system
• Misconfigured networking inside the OS
• Kernel panic
• Exhausted memory or disk space
• Failed startup scripts

Step-by-Step Fix

For System Status Check Failures

1. Stop and Start the Instance (Not Reboot)

A stop/start migrates the instance to new underlying hardware. A reboot does NOT.

# Stop the instance
aws ec2 stop-instances --instance-ids i-0abcd1234efgh5678

# Wait for it to stop
aws ec2 wait instance-stopped --instance-ids i-0abcd1234efgh5678

# Start it
aws ec2 start-instances --instance-ids i-0abcd1234efgh5678

# Wait for status checks to pass
aws ec2 wait instance-status-ok --instance-ids i-0abcd1234efgh5678

Warning: For instances with instance store volumes, stopping will erase all data on those volumes. EBS volumes are preserved.

Note: The public IP address will change unless you have an Elastic IP attached.

For Instance Status Check Failures

1. Check the System Log

aws ec2 get-console-output --instance-id i-0abcd1234efgh5678 --output text

Look for errors like:

• Kernel panic - not syncing
• FAILED mount /
• No space left on device
• error: networking service failed

2. Take a Screenshot

aws ec2 get-console-screenshot --instance-id i-0abcd1234efgh5678 --output text --query ImageData | base64 -d > screenshot.jpg

This shows exactly what's on the instance console -- useful for boot loops or login prompts.

3. Rescue the Instance Using a Helper

If the OS is corrupted, attach the root volume to a working instance:

# Stop the broken instance
aws ec2 stop-instances --instance-ids i-broken1234
aws ec2 wait instance-stopped --instance-ids i-broken1234

# Detach its root volume
aws ec2 detach-volume --volume-id vol-broken5678

# Attach to a healthy rescue instance
aws ec2 attach-volume --volume-id vol-broken5678 \
  --instance-id i-rescue9012 --device /dev/xvdf

Then SSH into the rescue instance and fix the issue:

# On the rescue instance
sudo mkdir /rescue
sudo mount /dev/xvdf1 /rescue

# Fix fstab if needed
sudo vi /rescue/etc/fstab

# Check and repair filesystem
sudo fsck /dev/xvdf1

# Check disk usage
sudo du -sh /rescue/* | sort -rh | head -10

Detach, reattach to the original instance, and start it.

4. If Disk Space Was the Problem

After recovery, clean up:

# Delete old logs
sudo journalctl --vacuum-size=100M

# Clean package cache
sudo apt-get clean  # Debian/Ubuntu
sudo yum clean all  # RHEL/Amazon Linux

# Find large files
sudo find / -xdev -type f -size +100M -exec ls -lh {} \;

Prevention Tips

• Enable Auto Recovery on critical instances: create a CloudWatch alarm that triggers recover action on StatusCheckFailed_System.
• Use EBS volumes, not instance store, for any data you can't lose.
• Attach Elastic IPs to instances you stop/start so the IP doesn't change.
• Monitor disk space with CloudWatch Agent and alert at 80% usage.
• Test user data scripts on disposable instances before applying to production.