User Tools

Site Tools


linux:ceph:tunning_recovery

Tunning CEPH recovery

Documentation
Name: Tunning CEPH recovery
Description: How to tune recovery options on CEPH for speed up recovery
Modification date :31/07/2018
Owner:dodger
Notify changes to:Owner
Tags:ceph, object storage
Scalate to:The_fucking_bofh

Instructions

Check actual values

ceph tell 'osd.*' config get osd_recovery_max_single_start 
ceph tell 'osd.*' config get osd_max_backfills 
ceph tell 'osd.*' config get osd_recovery_max_active 
ceph tell 'osd.*' config get osd_recovery_max_single_start 
ceph tell 'osd.*' config get osd_backfill_scan_max 
ceph tell 'osd.*' config get osd_backfill_scan_min 
ceph tell 'osd.*' config get osd_recovery_sleep_hdd

Change values

This will change the values on the running instances, without having to restart the OSD daemon:

ceph tell 'osd.*' config set osd_recovery_max_single_start 1000
ceph tell 'osd.*' config set osd_max_backfills 100
ceph tell 'osd.*' config set osd_recovery_max_active 300
ceph tell 'osd.*' config set osd_recovery_max_single_start 1000
ceph tell 'osd.*' config set osd_backfill_scan_max 51200
ceph tell 'osd.*' config set osd_backfill_scan_min 6400
ceph tell 'osd.*' config set osd_recovery_sleep_hdd 0

And add the options to the ceph.conf file under [osd] section (see Modifying ceph.conf):

osd recovery max single start = 1000
osd max backfills = 100
osd recovery max active = 300
osd recovery max single start = 1000
osd backfill scan max = 51200
osd backfill scan min = 6400
osd recovery sleep hdd = 0

Restart osd's

Not necessary using ceph tell command.

From osm-001 (with environment variables initialized):

for i in ${OSDSERVERS} ; do echo "###### ${i}" ;  ssh ${i} "sudo systemctl restart ceph-osd.target" ; sleep 5; done
linux/ceph/tunning_recovery.txt · Last modified: 2022/02/11 11:36 (external edit)