User Tools

Site Tools


ceph:troubleshooting:too_few_pgs_per_osd

[TROUBLESHOOT] Ceph too few pgs per osd

Documentation
Name: [TROUBLESHOOT] Ceph too few pgs per osd
Description: how to solve this “issue”
Modification date :13/06/2019
Owner:dodger@ciberterminal.net
Notify changes to:Owner
Tags:ceph, object storage

WARNING

This documents cover the TOO FEW PGs per OSD, not too many (documented here)

The error

ceph@avmlp-osm-001 ~/ceph-deploy $ sudo ceph health
HEALTH_WARN too few PGs per OSD (4 < min 30)

Official Documentation: http://docs.ceph.com/docs/master/rados/operations/health-checks/#many-objects-per-pg

The solution

The problem is that there's any pool with less PGs than configured as WARNING threshold.

ceph@avmlp-osm-001 ~/ceph-deploy $ sudo ceph osd pool ls detail
pool 1 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode warn last_change 171 flags hashpspool stripe_width 0 application rgw
pool 2 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode warn last_change 173 flags hashpspool stripe_width 0 application rgw
pool 3 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode warn last_change 175 flags hashpspool stripe_width 0 application rgw
pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode warn last_change 176 flags hashpspool stripe_width 0 application rgw

Just change the pg_num, pgp_num values:

ceph osd pool set POOL_NAME pg_num_actual 500

All pool at once:

for i in $(ceph osd pool ls) ; do  ceph osd pool set ${i} pgp_num 60 ; done

WARNING a huge number of PGs in all the pools can lead in error:

Error ERANGE: pool id 4 pg_num 500 size 3 would mean 6000 total pgs, which exceeds max 5000 (mon_max_pg_per_osd 250 * num_in_osds 20)

See [TROUBLESHOOT] Error ERANGE: pool id X pg_num Y size Z would mean W total pgs, which exceeds

ceph/troubleshooting/too_few_pgs_per_osd.txt · Last modified: 2019/07/18 09:17 (external edit)