ZFS Quotas and Reservations

ZFS supports quotas and reservations at the file system level. You can use the quota property to set a limit on the amount of space a file system can use. In addition, you can use the reservation property to guarantee that some amount of space is available to a file system. Both properties apply to the dataset they are set on and all descendents of that dataset.

That is, if a quota is set on the tank/home dataset, the total amount of space used by tank/home and all of its descendents cannot exceed the quota. Similarly, if tank/home is given a reservation, tank/home and all of its descendents draw from that reservation. The amount of space used by a dataset and all of its descendents is reported by the used property.

In addition to the quota and reservation property, the refquota and refreservation properties are available to manage file system space without accounting for space consumed by descendents, such as snapshots and clones.

Consider the following points to determine which quota and reservations features might better manage your file systems:

The quota and reservation properties are convenient for managing space consumed by datasets.
The refquota and refreservation properties are appropriate for managing space consumed by datasets and snapshots.
Setting refquota or refreservation higher than quota or reservation have no effect. If you set the quota or refquota properties, operations that try to exceed either value fail. It is possible to a exceed a quota that is greater than refquota. If some snapshot blocks are dirtied, you might actually exceed the quota before you exceed the refquota.

For more information, see the examples below.

Setting Quotas on ZFS File Systems

ZFS quotas can be set and displayed by using the zfs set and zfs get commands. In the following example, a quota of 10 Gbytes is set on tank/home/bonwick.

# zfs set quota=10G tank/home/bonwick
# zfs get quota tank/home/bonwick
NAME              PROPERTY      VALUE                      SOURCE
tank/home/bonwick quota         10.0G                      local

ZFS quotas also impact the output of the zfs list and df commands. For example:

# zfs list
NAME                   USED  AVAIL  REFER  MOUNTPOINT
tank/home             16.5K  33.5G  8.50K  /export/home
tank/home/bonwick     15.0K  10.0G  8.50K  /export/home/bonwick
tank/home/bonwick/ws  6.50K  10.0G  8.50K  /export/home/bonwick/ws
# df -h /export/home/bonwick
Filesystem             size   used  avail capacity  Mounted on
tank/home/bonwick       10G     8K    10G     1%    /export/home/bonwick

Note that although tank/home has 33.5 Gbytes of space available, tank/home/bonwick and tank/home/bonwick/ws only have 10 Gbytes of space available, due to the quota on tank/home/bonwick.

You cannot set a quota to an amount less than is currently being used by a dataset. For example:

# zfs set quota=10K tank/home/bonwick
cannot set quota for 'tank/home/bonwick': size is less than current used or 
reserved space

You can set a refquota on a dataset that limits the amount of space that the dataset can consume. This hard limit does not include space that is consumed by snapshots and clones. For example:

# zfs set refquota=10g students/studentA
# zfs list
NAME                USED  AVAIL  REFER  MOUNTPOINT
profs               106K  33.2G    18K  /profs
students           57.7M  33.2G    19K  /students
students/studentA  57.5M  9.94G  57.5M  /students/studentA
# zfs snapshot students/studentA@today
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                     106K  33.2G    18K  /profs
students                 57.7M  33.2G    19K  /students
students/studentA        57.5M  9.94G  57.5M  /students/studentA
students/studentA@today      0      -  57.5M  -

For additional convenience, you can set another quota on a dataset to help manage the space that is consumed by snapshots. For example:

# zfs set quota=20g students/studentA
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                     106K  33.2G    18K  /profs
students                 57.7M  33.2G    19K  /students
students/studentA        57.5M  9.94G  57.5M  /students/studentA
students/studentA@today      0      -  57.5M  -

In this scenario, studentA can bump into the refquota (10 Gbytes) hard limit and remove files to recover even if snapshots exist.

In the above example, the smaller of the two quotas (10 Gbytes versus 20 Gbytes) is displayed in the zfs list output. To see the value of both quotas, use the zfs get command. For example:

# zfs get refquota,quota students/studentA
NAME               PROPERTY  VALUE              SOURCE
students/studentA  refquota  10G                local
students/studentA  quota     20G                local

Setting Reservations on ZFS File Systems

A ZFS reservation is an allocation of space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve space for a dataset if that space is not currently available in the pool. The total amount of all outstanding unconsumed reservations cannot exceed the amount of unused space in the pool. ZFS reservations can be set and displayed by using the zfs set and zfs get commands. For example:

# zfs set reservation=5G tank/home/moore
# zfs get reservation tank/home/moore
NAME             PROPERTY      VALUE                      SOURCE
tank/home/moore  reservation   5.00G                      local

ZFS reservations can affect the output of the zfs list command. For example:

# zfs list
NAME                   USED  AVAIL  REFER  MOUNTPOINT
tank/home             5.00G  33.5G  8.50K  /export/home
tank/home/moore       15.0K  10.0G  8.50K  /export/home/moore

Note that tank/home is using 5 Gbytes of space, although the total amount of space referred to by tank/home and its descendents is much less than 5 Gbytes. The used space reflects the space reserved for tank/home/moore. Reservations are considered in the used space of the parent dataset and do count against its quota, reservation, or both.

# zfs set quota=5G pool/filesystem
# zfs set reservation=10G pool/filesystem/user1
cannot set reservation for 'pool/filesystem/user1': size is greater than 
available space

A dataset can use more space than its reservation, as long as space is available in the pool that is unreserved and the dataset's current usage is below its quota. A dataset cannot consume space that has been reserved for another dataset.

Reservations are not cumulative. That is, a second invocation of zfs set to set a reservation does not add its reservation to the existing reservation. Rather, the second reservation replaces the first reservation.

# zfs set reservation=10G tank/home/moore
# zfs set reservation=5G tank/home/moore
# zfs get reservation tank/home/moore
NAME             PROPERTY      VALUE                      SOURCE
tank/home/moore  reservation   5.00G                      local

You can set a refreservation to guarantee space for a dataset that does not include space consumed by snapshots and clones. The refreservation reservation is accounted for in the parent datasets' space used, and counts against the parent datasets' quotas and reservations. For example:

# zfs set refreservation=10g profs/prof1
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                    10.0G  23.2G    19K  /profs
profs/prof1                10G  33.2G    18K  /profs/prof1

You can also set a reservation on the same dataset to guarantee dataset space and snapshot space. For example:

# zfs set reservation=20g profs/prof1
# zfs list
NAME                      USED  AVAIL  REFER  MOUNTPOINT
profs                    20.0G  13.2G    19K  /profs
profs/prof1                10G  33.2G    18K  /profs/prof1

Regular reservations are accounted for in the parent's used space.

In the above example, the smaller of the two quotas (10 Gbytes versus 20 Gbytes) is displayed in the zfs list output. To see the value of both quotas, use the zfs get command. For example:

# zfs get reservation,refreserv profs/prof1
NAME         PROPERTY        VALUE        SOURCE
profs/prof1  reservation     20G          local
profs/prof1  refreservation  10G          local

If refreservation is set, a snapshot is only allowed if enough free pool space exists outside of this reservation to accommodate the current number of referenced bytes in the dataset.