Fix: Linux Disk Full / No Inodes Left

!Symptoms

• Commands fail with 'No space left on device'
• Applications cannot write to disk, logging stops
• df shows disk at 100% or 'No space left' despite df showing free space (inode issue)
• System becomes unresponsive or services crash

?Root Causes

• Log files growing unbounded without rotation
• Old package caches, temp files, or core dumps consuming space
• Deleted files still held open by running processes (space not freed)
• Too many small files exhausting inodes (common with mail queues, session files)
• Docker or container runtime consuming disk in /var/lib/docker

#Diagnosis Steps

1. 1Check disk usage: `df -h` for space, `df -i` for inodes
2. 2Find large files: `du -sh /* | sort -rh | head -20`
3. 3Find large directories: `du -sh /var/log/* | sort -rh | head -10`
4. 4Check for deleted but open files: `lsof +L1`
5. 5Check inode-heavy directories: `find / -xdev -type d -exec sh -c 'echo "$(find {} -maxdepth 1 | wc -l) {}"' \; | sort -rn | head -20`

>Fix

1. 1Delete old logs: `find /var/log -name '*.gz' -mtime +30 -delete`
2. 2Clean package cache: `apt clean` or `yum clean all`
3. 3Restart processes holding deleted files to free space: use `lsof +L1` to identify them
4. 4Truncate a large log file without stopping the service: `> /var/log/large.log`
5. 5For inodes: find and remove the directory with millions of small files

*Prevention

• Configure logrotate for all application and system logs
• Set up disk usage monitoring and alerts at 80% and 90%
• Use separate partitions for /var/log, /tmp, and data directories
• Schedule regular cleanup of tmp files and package caches via cron
• Monitor inode usage in addition to disk space

Related Error Messages