There are several reasons why a filesystem gets full. An important thing to consider is how you set up your filesystems during installation. One needs to take care how much space is used for each filesystem and think ahead.
With forward thinking, it is less likely that your filesystems will get full, but will not prevent a filesystem getting full. This post will show the most common reasons why a filesystem may become full and how to handle them. The following errors could be seen when a filesystem is full:
On the system console:
vmunix : file system full WARNING: file system full NOTICE: alloc: file system full no space left on device
WARNING: /: filesystem full
Resolving filesystem full situations
This post is written mainly to deal with the OS filesystems (such as root, var, and usr), but it can be utilized to troubleshoot other filesystems. There are many ways of finding what’s filling up a filesystem, which can sometimes be a difficult process. One problem is that a filesystem can be filled up by one or few very large files (which is generally easy to find) or by thousands of smaller files (which can be difficult to find and pinpoint the cause).
First, you need to figure out which files are filling up your filesystem. A very useful way to list the size of files in a filesystem is with the du command. The following example lists files from largest to smallest on the root filesystem:
$ du -akd / | sort -nr
-d – keeps du from crossing partition boundaries.
-a – tells du to report file sizes (without this option du just reports the amount of space used in each directory.
-k – tells du to report in terms of kilobytes rather than 512-byte blocks. On Solaris 9 or later replace k with h if you prefer human-readable output, that is output in terms of kilobytes, megabytes or gigabytes depending on the number reported.
-nr – This sort option puts the files in reverse numerical order.
Of course, this can be used on filesystems other than root, just substitute the required path for / in the “du” command. The command “du -skd” summarizes the number of kilobytes used for a filesystem. If this is different from what is reported by the df -k command.
Standard filesystems to look at first will be:
If /tmp is full or contains large files, a reboot will clean this directory. A default Solaris installation shares the diskspace for /tmp and swap as you can see in the output of the df command.
Large files may appear here when trying to write to a device using the incorrect device name. For example /dev/rmt/o (letter ‘o’) instead of /dev/rmt/0 (digit ‘zero’ for a tape drive. This is a very common problem if the machine does not have a tape drive attached and someone uses a tape command like tar or ufsdump. That will just create a large file in /dev/rmt/. So be sure to check the /dev directory for actual links, not files.
Look for core files. Check /.wastebasket and /lost+found directory for large files. Check for a .CPR file in root, this is put there by power suspend/resume software.
Third party packages sometimes leave tar files in /var/sadm/pkg directory. If /var is full (and is a separate filesystem) or /var directory is the one we determined is using up most space in the root, check the following.
Clearing out (but NOT deleting; the files should be truncated to zero length) the following files might gain you some space. Use caution here because you will lose various log information. For example, the utmp[x] and wtmp[x] files contain user access and accounting information:
/var/cron/log /var/spool/lp/logs /var/adm/utmp /var/adm/utmpx /var/adm/wtmp /var/adm/wtmpx /var/log/syslog* /var/adm/messages.*
To zero out a file, use the below command.
# cat /dev/null > filename
– Check /var/saf – check for _log and in tcp and zsmon directories. There will be _log files – you can zero them out with “cat /dev/null > filename”.
– If your system is being used as a printer host, check /var/lp/logs for files, they can be removed if they have been printed or left over from system crashes or printer problems. Check /var/preserve. Check /var/spool/* directory. Subdirectories like “lp” or “mqueue” are used for spooling.
– Check /var/crash for any system cores. Also check /var/tmp for files not needed. /var/tmp is not cleaned up with a reboot.
– A word of caution regarding the /var/sadm directory. This directory contains package and patch information and generally should not be touched.
– Also it could be that you are running out of inodes and are getting the message file system full.
# /usr/bin/df -F ufs -o i [directory | special]