Table of Contents
[TROUBLESHOOT] Ceph too few pgs per osd
Documentation | |
---|---|
Name: | [TROUBLESHOOT] Ceph too few pgs per osd |
Description: | how to solve this “issue” |
Modification date : | 13/06/2019 |
Owner: | dodger |
Notify changes to: | Owner |
Tags: | ceph, object storage |
Scalate to: | Thefuckingbofh |
WARNING
This documents cover the TOO FEW PGs per OSD, not too many (documented here)
The error
ceph@avmlp-osm-001 ~/ceph-deploy $ sudo ceph health HEALTH_WARN too few PGs per OSD (4 < min 30)
Official Documentation: http://docs.ceph.com/docs/master/rados/operations/health-checks/#many-objects-per-pg
So “the problem” is that there's any pool with less PGs (pg_num
) that the configured in the warning threshold.
The solutions
There are 2 solutions for this warning:
- Increase the
pgnum
in autoscale mode.or
pgnum_min - Change the warning threshold.
Increase pg_num/pg_num_min
pg_num
See which pool is the one with less PG's than expected:
ceph osd pool ls detail
Just change the pgnum
values:
,
pgpnum
ceph osd pool set ${POOL_NAME} pg_num 500 ceph osd pool set ${POOL_NAME} pgp_num 500
All pool at once (TAKE CARE with this, I've wrote an echo
to avoid problems, think twice before executing):
for i in $(ceph osd pool ls) ; do echo ceph osd pool set ${i} pg_num 60 ; done
WARNING a huge number of PGs in all the pools can lead in error:
Error ERANGE: pool id 4 pg_num 500 size 3 would mean 6000 total pgs, which exceeds max 5000 (mon_max_pg_per_osd 250 * num_in_osds 20)
See [TROUBLESHOOT] Error ERANGE: pool id X pg_num Y size Z would mean W total pgs, which exceeds
pg_num_min
That's for pgautoscalemode on
:
ceph osd pool set ${POOL_NAME} pg_num_min ${MINPGS}
If pool actual pgnum
also:
is lower than
pgnummin you'll have to change
pgnum
ceph osd pool set ${POOL_NAME} pg_num ${MINPGS} ceph osd pool set ${POOL_NAME} pg_num_min ${MINPGS}
Change the warning threshold
Add the following to ceph.conf
under [global]
section:
mon pg warn min per osd = 8
and restart mon
and mgr
(mon to apply the changes, mgr for the monitoring).