• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer navigation

The Geek Diary

  • OS
    • Linux
    • CentOS/RHEL
    • Solaris
    • Oracle Linux
    • VCS
  • Interview Questions
  • Database
    • oracle
    • oracle 12c
    • ASM
    • mysql
    • MariaDB
  • DevOps
    • Docker
    • Shell Scripting
  • Big Data
    • Hadoop
    • Cloudera
    • Hortonworks HDP

How To Delete Files on a ZFS Filesystem that is 100% Full

by admin

When a ZFS file system has no space left then the deletion of files can fail with “disk quota exceeded“. The post provides different ways to create free space to overcome the situation.

1. Truncating files

If the files can not be removed directly, we can first truncate them and then delete them.

# cat /dev/null > /file/to/delete
# rm /file/to/delete

Alternatives: there are other options to free up space in the zpool, e.g.
1. increase the quota if there is space in the zpool left
2. Shrink the size of a zvol
3. temporarily destroy a dump device (if the rpool is affected)
4. delete unused snapshots
5. increase the space in the zpool by enlarging a vdev or adding a vdev
6. Temporarily decrease refreservation of a ZVol
7. Rolling back the log

2. Increasing the quota

You can check the current quota with

# zfs get quota {filesystem}
# zfs get refquota {filesystem}

If the quota is set then you can increase it with

# zfs set quota={value} {dataset}
# zfs set refquota={value} {dataset}

3. Shrink the size of a zvol

If the zvol of the affected zpool has some space left then it can be shrinked.

# zfs set volsize=[newsize] ZPOOL/ZVOL

For the rpool and its dump zvol it looks like this:

# zfs set volsize=[newsize] rpool/dump

4. Destroying a dump device on an rpool (or an unused zvol)

The rpool’s dump device can temporarily be destroyed, as it is used for storing a livedump (“savecore -L”) or a crash dump in case of a panic.

# dumpadm
Dump content      : kernel with ZFS metadata
Dump device       : /dev/zvol/dsk/rpool/dump (dedicated)
Savecore directory: /var/crash
Savecore enabled  : yes
Save compressed   : on

You can inspect all zvols with

# zfs list -t volume
NAME            USED  AVAIL  REFER  MOUNTPOINT
rpool/dump     5.16G  11.4G  4.00G  -
rpool/swap     2.06G  10.3G  2.00G  -
rpool/testvol   103M  10.3G  8.13M  -
# zfs list rpool/dump
NAME         USED  AVAIL  REFER  MOUNTPOINT
rpool/dump  5.16G  11.4G  4.00G  -

Please note the volsize for the re-creation of the zvol later on; in this case 4 GB of space can be created by destroying the zvol.

# zfs destroy rpool/dump

Later you can re-create the zvol with (note that the size of 4 GB is used here according to the example; specify the volsize you saw in your output of “zfs get volsize rpool/dump”):

# zfs create -V 4g rpool/dump

5. Delete unused snapshots

List the snapshots with

# zfs list -t snapshot

You can sort the output; here the “used” property is used:

# zfs list -t snapshot -S used

You can see the contents of a snapshot; there is a special directory “.zfs/snapshot” directory in the root of the file system. By default the directoy “.zfs” is hidden. You need to change the property “snapdir” to see the directory:

# zfs set snapdir=visible {filesystem}

Now you can change to the directoy /{filesystem}/.zfs/snapshot with cd(1) and inspect the contents. Then decide which snapshot you do not need and delete it with

# zfs destroy {snapshot}

6. Increase the space in the zpool by enlarging a vdev or adding a vdev

You can add another vdev to the zpool to make it bigger. If your zpool consists of raidzN vdevs please make sure that you add another raidzN vdevs. If you would erroneously add a single vdev then your zpool is no longer fully redundant; you could attach another vdev to have a mirror added to the raidzN vdevs and have another type of redundancy. But if you make a mistake then this can just be corrected by re-creating the whole zpool.

You can grow a vdev by growing the underlying LUN or make the partition larger. Please make sure that the following property is set for the zpool:

# zpool get autoexpand {zpool}
# zpool set autoexpand=on {zpool}

This way ZFS will detect changes in the size of the used storage devices and will propagate the change to the vdev.

7. Temporarily decrease refreservation of a ZVol

Temporarily decrease the refreservation of any zvols that are not using all of their reserved space. This frees space within the zpool to allow a more permanent solution to be implemented, before the refreservation is restored to its original value.

List the zvols:

# zfs list -t vol
NAME        PROPERTY        VALUE  SOURCE
rpool/dump  referenced      4.00G  -
rpool/dump  refreservation  5.16G  local

Select a zvol and display its current refreserv and usedbyrefreserv:

# zfs get refreservation,usedbyrefreservation rpool/dump
NAME        PROPERTY              VALUE  SOURCE
rpool/dump  refreservation        5.16G  local
rpool/dump  usedbyrefreservation  1.16G  -

Decrease the refreserv:

# zfs set refreservation=5.0G rpool/dump

Note: ‘refreservation’ may be greater than necessary. You can use ‘auto’ to reserve enough space for a non-sparse volume. After you successfully deleted the data restore the refreservation back to the original value, here:

# zfs set refreservation=5.16G rpool/dump

Note: ‘refreservation’ may be greater than necessary. You can use ‘auto’ to reserve enough space for a non-sparse volume.

8. Rolling back the log

If the import of a zpool fails due to “out of space” then you can import with rolling the log back to the previous commit to the log; however all updates to the filesystem after that commit was made are lost. You can achieve this by

# zpool import -F ZPOOLNAME

It will tell you the point in time to which the log was rolled back, e.g.:

# zpool import -F rpool

Pool rpool returned to its state as of Thu Jun 09 07:12:01 2016. If you just want to check whether this is possible then you can do

# zpool import -Fn ZPOOLNAME

Filed Under: Solaris, ZFS

Some more articles you might also be interested in …

  1. How to troubleshoot Solaris 10 SMF (Service Management Facility) related issues
  2. How to Configure a Solaris 10 Jumpstart server and client [SPARC]
  3. What are the Solaris process tools
  4. Solaris Volume Manager (SVM) : How to Use Mirrors to Roll Back System Changes
  5. SVM : How to set boot device at OBP for mirrored root disk
  6. How to configure NTP client in Solaris 8,9,10 and non-global zones
  7. How to configure rsyslog on Solaris 11.1 to send messages to a remote host using TCP
  8. Solaris ZFS : How to Create and Manage Mirrored Storage Pools
  9. How to Set the TimeZone in Solaris 10,11
  10. What is SUID, SGID and Sticky bit ?

You May Also Like

Primary Sidebar

Recent Posts

  • What are /dev/zero and /dev/null files in Linux
  • grpck command – Remove corrupt or duplicate entries in the /etc/group and /etc/gshadow files.
  • xxd command – Expressed in hexadecimal form
  • sesearch: command not found

© 2022 · The Geek Diary

  • Archives
  • Contact Us
  • Copyright