User Tools

Site Tools


ceph:extending_cluster_add_mons

Extending the cluster: Adding monitors

Documentation
Name: Extending the cluster: Adding monitors
Description: How to add more nodes to a running cluster
Modification date :19/07/2019
Owner:dodger@ciberterminal.net
Notify changes to:Owner
Tags:ceph, object storage

Pre-Requisites

Variables used in this documentation

Name Description Sample
${THESERVER} Variable used as salt target, it can be a mask of serves (see sample)
export THESERVER="pro-cephm-00[56]*"
${NEWSERVERS} Variable used for clonewars target and ceph-deploy
export NEWSERVERS="pro-cephm-005 pro-cephm-006"
${VMNAMESTART} Variable used to perform a regex in salt execution, it will match the environment (pro, demoenv …)
export VMNAMESTART="pro"

Instructions

Modify ceph.conf

You must add the new monitors to ceph.conf:

[mon.pro-cephm-005]
host = pro-cephm-005.ciberterminal.net
addr = 10.20.54.55
public_addr = 10.20.54.55:6789
 
[mon.pro-cephm-006]
host = pro-cephm-006.ciberterminal.net
addr = 10.20.54.56
public_addr = 10.20.54.56:6789

And populate to all the cluster:

ceph-deploy --overwrite-conf config push ${ALLSERVERS}

Then restart the monitors (yes, is a salt command):

salt "${THESERVER}" service.restart ceph-mon.target

Method 1: ceph-deploy

As seen here:

ceph-deploy mon create ${NEWSERVERS}

Method 2: Manual (as official instructions)

As seen here:

As ceph user, in the admin node:

export TMPDIR=~/joining_ceph
mkdir ${TMPDIR}
cd  ${TMPDIR}
sudo ceph auth get mon. -o keyfile.txt
sudo ceph mon getmap -o mapfile.bin
for i in ${NEWSERVERS} ; do scp -r "${TMPDIR}" ${i}:${TMPDIR} ; done

Then in each new node:

export TMPDIR=~/joining_ceph
sudo ceph-mon -i $(hostname) --mkfs --monmap ${TMPDIR}/mapfile.bin --keyring ${TMPDIR}/keyfile.txt

Method 3: Fully manual with fsid (the working one)

A little modification of the manual documentation, more info here

As ceph user, in the admin node:

export TMPDIR=~/joining_ceph
mkdir ${TMPDIR}
cd  ${TMPDIR}
sudo ceph auth get mon. -o keyfile.txt
sudo ceph mon getmap -o mapfile.bin
for i in ${NEWSERVERS} ; do scp -r "${TMPDIR}" ${i}:${TMPDIR} ; done

Then in each new node:

export TMPDIR=~/joining_ceph
sudo ceph-mon -i $(hostname) --mkfs --monmap ${TMPDIR}/mapfile.bin --keyring ${TMPDIR}/keyfile.txt
ceph/extending_cluster_add_mons.txt · Last modified: 2019/07/23 11:59 by dodger