IBM FlashSystem SCSI unmap

XorMon shows since its version 2.5.0, for every volume, pool and child pool of IBM SAN Volume Controller, Storwize and FlashSystem, how many SCSI unmap operations the hosts sent, how much data they unmapped and how long the operations took.

Overview

Hosts send SCSI unmap commands (also known as trim or discard) to tell the storage system that blocks are no longer in use, for example after files were deleted. Typical sources are the Linux fstrim command or the discard mount option, Windows Optimize Drives and automatic space reclamation in hypervisors such as VMware VMFS.

On thin-provisioned and compressed volumes and in data reduction pools, the storage system can then release the capacity behind the unmapped blocks. The unmap metrics show:

  • whether the hosts send unmap commands at all, and when,
  • how much data they unmap,
  • how long the storage system takes to process the commands, which matters during large reclamation runs.

Metrics

The storage reports cumulative unmap counters per volume and node in its I/O statistics files. XorMon takes the increase over each collection interval, sums it over the nodes and shows it as a rate, so 0 means no unmap command in that interval.

XorMon metric IBM counter Unit Meaning
Unmap IO uo IO/s Unmap operations processed per second
Unmap Data ub B/s Amount of data unmapped per second (blocks × block size)
Unmap Latency ul / uo ms Average response time of an unmap operation, from receiving the SCSI unmap command to its successful completion

Where to find the graphs

Page Tab Subtabs What it shows
IBM FlashSystem → Volume → Totals Unmap IO, Data, Latency All volumes of the device: IO and Data stacked per volume, Latency as a line per volume
Single volume Unmap IO, Data, Latency One volume
IBM FlashSystem → Pool → Totals Unmap IO, Data, Latency All pools of the device
Single pool Unmap IO, Data, Latency, Volume IO, Volume Data, Volume Latency The pool itself, and the volumes in the pool (to find which volumes generate the unmap load)
IBM FlashSystem → Child Pool → Totals Unmap IO, Data, Latency All child pools of the device
Single child pool Unmap IO, Data, Latency, Volume IO, Volume Data, Volume Latency The child pool itself, and the volumes in the child pool
IBM FlashSystem → Totals Unmap Pool IO, Pool Data, Pool Latency, Child Pool IO, Child Pool Data, Child Pool Latency, Volume IO, Volume Data, Volume Latency All pools, child pools and volumes of the device in one place

How pool values are calculated

The storage system reports unmap statistics only per volume. XorMon calculates the pool and child pool values from the volumes they contain:

  • Unmap IO and Unmap Data: the sum over the volumes of the pool.
  • Unmap Latency: the average of the volumes' latency weighted by their unmap IO, so a volume with many unmap operations counts more than one with a few. When no volume of the pool sends unmap commands, the latency is 0 ms, the same as for a volume.
  • Mirrored volumes with copies in two pools (for example in a stretched cluster): the unmap is counted in full in both pools, because it applies to both copies, just like writes.

How to read the graphs

Unmap commands come only when hosts release space, so most of the time the graphs stay at 0 and show spikes after deletions or scheduled trim runs.

  • No unmap at all: the hosts do not send unmap commands. Check whether unmap or discard is enabled on the host, in the file system or in the hypervisor.
  • Regular spikes: scheduled reclamation, for example a weekly fstrim or a hypervisor reclamation job. The spike shows how much data was released and on which volumes.
  • Unmap and capacity: compare the Unmap Data graph with the volume and pool capacity graphs. In data reduction pools the released capacity is returned to the pool. In standard pools the used capacity of a volume may not go down after an unmap; it can be reclaimed for example by creating a new volume copy and removing the old one.
  • High unmap latency: large reclamation runs can load the system. Compare with the volume Latency graphs of the same time and, if normal IO is affected, schedule the trim runs outside peak hours.

Limitations

  • The graphs stay empty on systems whose firmware does not report the unmap counters in its I/O statistics.
  • The values are averages over the collection interval; a short trim burst shows a lower rate spread over the whole interval.
  • The worst (peak) unmap response time and node-level unmap statistics are not shown.
  • The counters start from zero when a node restarts; the first interval after the restart shows the operations counted since then.

More detailed information about SCSI unmap you can find in IBM redbook.

Last updated: