My disk is filling up. How can I find the culprit?

There are a few useful tools available to help you find what is using up the most disk space on your Linode. The first step I would recommend is taking a look at your Linode's df -h and df -i output. df -h will tell you how much disk space is being used, while df -i will tell you how many inodes, or how many files are on the Linode.

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        20G   16G  3.4G  82% /
devtmpfs        488M     0  488M   0% /dev
tmpfs           490M  176K  490M   1% /dev/shm
tmpfs           490M   16M  475M   4% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           490M     0  490M   0% /sys/fs/cgroup
/dev/sdc         20G  204M   19G   2% /mnt/readspeed
tmpfs            98M   32K   98M   1% /run/user/110
tmpfs            98M     0   98M   0% /run/user/1000

df -i
Filesystem      Inodes  IUsed   IFree IUse% Mounted on
/dev/root      1228608 170095 1058513   14% /
devtmpfs        124707   1387  123320    2% /dev
tmpfs           125413      6  125407    1% /dev/shm
tmpfs           125413   1275  124138    2% /run
tmpfs           125413      4  125409    1% /run/lock
tmpfs           125413     17  125396    1% /sys/fs/cgroup
/dev/sdc       1310720     13 1310707    1% /mnt/readspeed
tmpfs           125413     18  125395    1% /run/user/110
tmpfs           125413      4  125409    1% /run/user/1000

In the above example, 82% of my Linode's disk space is being used up, but only 14% of the inodes. This indicates to me that large file(s) are responsible for the majority of the disk space on my Linode. From there, I can begin searching.

Using DU

One useful command you can run is a variation of duwhich will tell you the disk usage of each file recursively for all directories. The command I would generally recommend running is sudo du -ahx / | sort -rh | head -n 20. This is a quick and easy way to find the largest file on your Linode as it scans the entire root directory and provides you with the largest folder and file:

sudo du -ahx / | sort -rh | head -n 20
16G    /
11G    /home/scrane/file.txt
11G    /home/scrane
11G    /home
3.1G    /usr
1.7G    /usr/lib
1010M    /usr/share
977M    /home/scrane/1gb.file
975M    /var
866M    /usr/lib/x86_64-linux-gnu
633M    /var/lib
331M    /var/cache
309M    /var/cache/apt
279M    /usr/bin
249M    /var/lib/clamav
237M    /usr/lib/libreoffice
216M    /var/cache/apt/archives
204M    /usr/share/fonts
185M    /usr/lib/libreoffice/program
175M    /var/lib/apt/lists

Based on the above, I have two large files I would want to consider removing: the file.txt and the 1gb.file.

NCDU

Another really useful tool to help you better visualize your filesystem is ncdu. This isn't included in Linode's base images, however you can quickly install it on your Linode using apt-get or yum. ncdu is a visual, interactive way to take a look at your Linode's filesystem structure and allows you to browse to find your largest folders.

sudo ncdu /
ncdu 1.11 ~ Use the arrow keys to navigate, press ? for help
--- / ---------------------------------------------------------------------------------------------------------------------------------------------------
   11.0 GiB [##########] /home
    3.0 GiB [##        ] /usr
  974.4 MiB [          ] /var
  160.0 MiB [          ] /mnt
   95.8 MiB [          ] /lib
   31.7 MiB [          ] /boot
.  15.2 MiB [          ] /run
   12.8 MiB [          ] /etc
   12.6 MiB [          ] /bin
   12.4 MiB [          ] /sbin
  176.0 KiB [          ] /dev
  140.0 KiB [          ] /opt
   52.0 KiB [          ] /tmp
   40.0 KiB [          ] /root
e  16.0 KiB [          ] /lost+found
    8.0 KiB [          ] /media
    8.0 KiB [          ] /snap
    4.0 KiB [          ] /lib64
e   4.0 KiB [          ] /srv
.   0.0   B [          ] /proc
    0.0   B [          ] /sys
@   0.0   B [          ]  initrd.img
@   0.0   B [          ]  vmlinuz

Total disk usage:  15.3 GiB  Apparent size:  15.2 GiB  Items: 291507

Looking at that output, I can see that /home is using up the most disk space, followed by /usr. If I press the right arrow key, the screen looks like this:

--- /home -----------------------------------------------------------------------------------------------------------------------------------------------
 /..
   11.0 GiB [##########] /scrane
Total disk usage:  11.0 GiB  Apparent size:  11.0 GiB  Items: 737

Then again to go into /home/scrane.

--- /home/scrane ------------------------------------------------------------------------------------------------------------------------------------
                         /..
   10.0 GiB [##########]  file.txt
    1.0 GiB [          ]  1gb.file
   12.5 MiB [          ] /nano-2.9.3
    3.6 MiB [          ]  mtr-backup.log
    2.7 MiB [          ]  nano-2.9.3.tar.gz
    1.5 MiB [          ]  mtr.log
  876.0 KiB [          ] /.vnc
  232.0 KiB [          ]  cacert.pem.1
  136.0 KiB [          ] /.cache
   32.0 KiB [          ]  .bash_history
   12.0 KiB [          ] /.dbus
   12.0 KiB [          ] /.ssh
    8.0 KiB [          ] /test
e   4.0 KiB [          ] /.nano
    4.0 KiB [          ]  .bashrc
    4.0 KiB [          ]  .profile
    4.0 KiB [          ]  count.sh
    4.0 KiB [          ]  .bash_logout
    4.0 KiB [          ]  .wget-hsts
    4.0 KiB [          ]  test.zip
    4.0 KiB [          ]  .selected_editor
    4.0 KiB [          ]  .Xauthority
    4.0 KiB [          ]  .lesshst
    4.0 KiB [          ]  test.txt
    0.0   B [          ]  cacert.pem
    0.0   B [          ]  .sudo_as_admin_successful
Total disk usage:  11.0 GiB  Apparent size:  11.0 GiB  Items: 736

If I scroll down to file.txt then press d, then highlight yes and press enter I will delete the file and free up 10GB on the Linode.

--- /home/scrane ------------------------------------------------------------------------------------------------------------------------------------
                         /..
    1.0 GiB [##########]  1gb.file
   12.5 MiB [          ] /nano-2.9.3
    3.6 MiB [          ]  mtr-backup.log
    2.7 MiB [          ]  nano-2.9.3.tar.gz
    1.5 MiB [          ]  mtr.log
  876.0 KiB [          ] /.vnc
  232.0 KiB [          ]  cacert.pem.1
  136.0 KiB [          ] /.cache
   32.0 KiB [          ]  .bash_history
   12.0 KiB [          ] /.dbus
   12.0 KiB [          ] /.ssh
    8.0 KiB [          ] /test
e   4.0 KiB [          ] /.nano
    4.0 KiB [          ]  .bashrc
    4.0 KiB [          ]  .profile
    4.0 KiB [          ]  count.sh
    4.0 KiB [          ]  .bash_logout
    4.0 KiB [          ]  .wget-hsts
    4.0 KiB [          ]  test.zip
    4.0 KiB [          ]  .selected_editor
    4.0 KiB [          ]  .Xauthority
    4.0 KiB [          ]  .lesshst
    4.0 KiB [          ]  test.txt
    0.0   B [          ]  cacert.pem
    0.0   B [          ]  .sudo_as_admin_successful
Total disk usage:   1.0 GiB  Apparent size:   1.0 GiB  Items: 735

Looking For the Most Number of Files

But what if you're looking for the folder that has the most number of files? This could be important if you are hitting one of the limitations of the Linode Backup Service or if your Linode is running out of inodes. One way to do this is with another modification to the du command.

du --inodes -S / | sort -rh | sed -n \
    '1,50{/^.\{71\}/s/^\(.\{30\}\).*\(.\{37\}\)$/\1...\2/;p}'

That will print you an output that looks as follows:

sudo du --inodes -S / | sort -rh | sed -n '1,50{/^.\{71\}/s/^\(.\{30\}\).*\(.\{37\}\)$/\1...\2/;p}'
7228    /var/lib/dpkg/info
3296    /usr/share/app-install/desktop
2327    /usr/share/man/man3
2104    /usr/share/app-install/icons
1922    /usr/share/man/man1
1813    /usr/bin
1646    /usr/lib/x86_64-linux-gnu
1555    /var/lib/app-info/icons/ubuntu-xenial-universe/64x64
825    /usr/share/icons/ubuntu-mono-light/status/22
825    /usr/share/icons/ubuntu-mono-dark/status/22
707    /usr/share/bash-completion/completions
702    /run/udev/data
691    /usr/share/man/man8
674    /dev
633    /sys/dev/char
633    /dev/char

You may want to pipe that output into less or use head -n 20 to see only the 20 top folders. Another way to do this would be to create a bash script.

#!/bin/bash

if [ $# -ne 1 ];then
  echo "Usage: `basename $0` DIRECTORY"
  exit 1
fi

echo "Wait a moment if you want a good top of the bushy folders..."

find "$@" -type d -print0 2>/dev/null | while IFS= read -r -d '' file; do 
    echo -e `ls -A "$file" 2>/dev/null | wc -l` "files in:\t $file"
done | sort -nr | head | awk '{print NR".", "\t", $0}'

exit 0

You'll want to save it to a file (for example, a file named "count.sh") and run it to search all of your directories with sudo bash count.sh /. You will get an output like

sudo bash count.sh /
Wait a moment if you want a good top of the bushy folders...
1.      7227 files in:  /var/lib/dpkg/info
2.      3295 files in:  /usr/share/app-install/desktop
3.      2326 files in:  /usr/share/man/man3
4.      2103 files in:  /usr/share/app-install/icons
5.      1921 files in:  /usr/share/man/man1
6.      1818 files in:  /usr/bin
7.      1813 files in:  /usr/share/doc
8.      1750 files in:  /usr/lib/x86_64-linux-gnu
9.      1554 files in:  /var/lib/app-info/icons/ubuntu-xenial-universe/64x64
10.      824 files in:   /usr/share/icons/ubuntu-mono-light/status/22

If you have any other suggestions or useful tips, please feel free to share them in response to this!

Hi there, this command not working for me:

du --inodes -S / | sort -rh | sed -n \
'1,50{/^.{71}/s/^(.{30}).*(.{37})$/\1…\2/;p}'

du: unrecognized option '--inodes'

Ubuntu 14.04
Any sugestion?

@them --

You write:

Hi there, this command not working for me:

Well, it's pretty obvious that the du(1) on Ubuntu 14.04 LTS (Trusty Tahr) doesn't have a --inodes option. See man du for more details. 14.04 is quite old (6 years…3 LTS releases older than the current one…and much older than @scrane's post) so it's likely that GNU coreutils has changed since 14.04 was released…or you haven't installed updates for awhile:

sudo apt-get update
sudo apt-get upgrade

-- sw