====== [TROUBLESHOOT] MDS cache too large ======

^  Documentation  ^|
^Name:| [TROUBLESHOOT] MDS cache too large |
^Description:| how to solve this "issue" |
^Modification date :|31/07/2019|
^Owner:|dodger|
^Notify changes to:|Owner |
^Tags:|ceph, object storage |
^Scalate to:|The_fucking_bofh|

====== The Warning ======


<code bash>
avmlp-osm-001 /home/ceph # ceph health detail
HEALTH_WARN 1 MDSs report oversized cache
MDS_CACHE_OVERSIZED 1 MDSs report oversized cache
    mdsavmlp-osfs-004.ciberterminal.net(mds.0): MDS cache is too large (2GB/1GB); 903372 inodes in use by clients, 0 stray files
</code>


====== The solution ======

Increase MDS cache, this is a **soft** warning, just reporting high memory cache usage by MDS.
\\

The setting must be changed on **every** MDS server, for example:
<code bash>
avmlp-osfs-004 /var/log/ceph # ceph daemon mds.avmlp-osfs-004.ciberterminal.net config get mds_cache_memory_limit                                                                                                                                                                         
{
    "mds_cache_memory_limit": "4294967296"
}
</code>

And from a monitor node, with the server list exported:
<code bash>
ceph@avmlp-osm-001 ~/ceph-deploy :( $ for i in ${MDSSERVERS} ; do echo "###### ${i}" ;  ssh ${i} "sudo ceph daemon mds.${i} config set mds_cache_memory_limit 4294967296 " ; done                                                                                                      
###### avmlp-osfs-002.ciberterminal.net
{
    "success": "mds_cache_memory_limit = '4294967296' "
}
###### avmlp-osfs-001.ciberterminal.net
{
    "success": "mds_cache_memory_limit = '4294967296' "
}
###### avmlp-osfs-004.ciberterminal.net
{
    "success": "mds_cache_memory_limit = '4294967296' "
}
###### avmlp-osfs-003.ciberterminal.net
{
    "success": "mds_cache_memory_limit = '4294967296' "
}
</code>


And add the corresponding option to ''ceph.conf'':
<code>
[mds]
# https://docs.ceph.com/ceph-prs/19690/cephfs/cache-size-limits/
mds cache memory limit = 4294967296
</code>
====== Official documentation ======

  * [[https://ceph.com/community/new-luminous-cephfs-metadata-server-memory-limits/]]