Block device tuning for Ceph (RBD)

Enabling Ceph and defining nova_libvirt_images_rbd_pool changes two libvirt configurations by default:

• hw_disk_discard: unmap
• disk_cachemodes: network=writeback

Setting hw_disk_discard to unmap in libvirt enables discard (sometimes called TRIM) support for the underlying block device. This allows reclaiming of unused blocks on the underlying disks.

Setting disk_cachemodes to network=writeback allows data to be written into a cache on each change, but those changes are flushed to disk at a regular interval. This can increase write performance on Ceph block devices.

You have the option to customize these settings using two Ansible variables (defaults shown here):

nova_libvirt_hw_disk_discard: 'unmap'
nova_libvirt_disk_cachemodes: 'network=writeback'

You can disable discard by setting nova_libvirt_hw_disk_discard to ignore. The nova_libvirt_disk_cachemodes can be set to an empty string to disable network=writeback.

The following minimal example configuration sets nova to use the ephemeral-vms Ceph pool. The following example uses cephx authentication, and requires an existing cinder account for the ephemeral-vms pool:

nova_libvirt_images_rbd_pool: ephemeral-vms
ceph_mons:
  - 172.29.244.151
  - 172.29.244.152
  - 172.29.244.153

If you have a different Ceph username for the pool, use it as:

cinder_ceph_client: <ceph-username>

• The Ceph documentation for OpenStack has additional information about these settings.
• OpenStack-Ansible and Ceph Working Example