Ciberterminal Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== [HOWTO] Mount CephFS ======

^  Documentation  ^|
^Name:| [HOWTO] Mount CephFS |
^Description:| [HOWTO] Mount CephFS|
^Modification date :|03/07/2019|
^Owner:|dodger|
^Notify changes to:|Owner|
^Tags:| ceph, object storage|
^Scalate to:|The_fucking_bofh|


====== Variables used in this documentation ======

^ Name ^ Description ^ Sample ^
| ''${CLIENTNAME}'' | name of the client for cephfs | <code bash>export CLIENTNAME="ftp"</code> | 
| ''${DATAPOOL}'' | Name of the data pool of ''cephfs'' | <code bash> export DATAPOOL="cephfs_data"</code> |


====== Instructions ======

===== on the client =====

Add the ''nautilus'' repo:
<code bash>
cat >/etc/yum.repos.d/ceph.repo<<EOF
[Ceph]
name=Ceph packages for
baseurl=http://download.ceph.com/rpm-nautilus/el7/\$basearch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc
priority=1

[Ceph-noarch]
name=Ceph noarch packages
baseurl=http://download.ceph.com/rpm-nautilus/el7/noarch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc
priority=1

[ceph-source]
name=Ceph source packages
baseurl=http://download.ceph.com/rpm-nautilus/el7/SRPMS
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc
priority=1
EOF
</code>

Install ''ceph-common'' package:
<code bash>
yum install ceph-common
</code>



Sample:
<code bash>
avmlp-sftp-001 /etc/yum.repos.d # yum install ceph-common
Loaded plugins: fastestmirror, versionlock
Loading mirror speeds from cached hostfile
 * base: ftp.csuc.cat
 * epel: mirror.airenetworks.es
 * extras: mirror.airenetworks.es
 * updates: ftp.csuc.cat
Resolving Dependencies
--> Running transaction check
---> Package ceph-common.x86_64 2:14.2.1-0.el7 will be installed

...
...

Dependencies Resolved

===============================================================================================================================================================================================================================================
 Package                                                            Arch                                                 Version                                                      Repository                                          Size
===============================================================================================================================================================================================================================================
Installing:
 ceph-common                                                        x86_64                                               2:14.2.1-0.el7                                               Ceph                                                18 M
Installing for dependencies:
 fuse-libs                                                          x86_64                                               2.9.2-11.el7                                                 base                                                93 k
 gperftools-libs                                                    x86_64                                               2.6.1-1.el7                                                  base                                               272 k
 leveldb                                                            x86_64                                               1.12.0-11.el7                                                epel                                               161 k
 libbabeltrace                                                      x86_64                                               1.2.4-3.el7                                                  epel                                               147 k
 libcephfs2                                                         x86_64                                               2:14.2.1-0.el7                                               Ceph                                               478 k
 libibverbs                                                         x86_64                                               17.2-3.el7                                                   base                                               246 k
 liboath                                                            x86_64                                               2.6.2-1.el7                                                  epel                                                51 k
 librabbitmq                                                        x86_64                                               0.8.0-2.el7                                                  base                                                37 k
 librados2                                                          x86_64                                               2:14.2.1-0.el7                                               Ceph                                               3.3 M
 libradosstriper1                                                   x86_64                                               2:14.2.1-0.el7                                               Ceph                                               340 k
 librbd1                                                            x86_64                                               2:14.2.1-0.el7                                               Ceph                                               1.6 M
 librdmacm                                                          x86_64                                               17.2-3.el7                                                   base                                                62 k
 librgw2                                                            x86_64                                               2:14.2.1-0.el7                                               Ceph                                               4.6 M
 lttng-ust                                                          x86_64                                               2.4.1-4.el7                                                  epel                                               176 k
 python-ceph-argparse                                               x86_64                                               2:14.2.1-0.el7                                               Ceph                                                36 k
 python-cephfs                                                      x86_64                                               2:14.2.1-0.el7                                               Ceph                                                89 k
 python-rados                                                       x86_64                                               2:14.2.1-0.el7                                               Ceph                                               190 k
 python-rbd                                                         x86_64                                               2:14.2.1-0.el7                                               Ceph                                               168 k
 python-rgw                                                         x86_64                                               2:14.2.1-0.el7                                               Ceph                                                76 k
 rdma-core                                                          x86_64                                               17.2-3.el7                                                   base                                                49 k
 userspace-rcu                                                      x86_64                                               0.7.16-1.el7                                                 epel                                                73 k

Transaction Summary
===============================================================================================================================================================================================================================================
Install  1 Package (+21 Dependent packages)

...
...
Installed:
  ceph-common.x86_64 2:14.2.1-0.el7                                                                                                                                                                                                            

Dependency Installed:
  fuse-libs.x86_64 0:2.9.2-11.el7     gperftools-libs.x86_64 0:2.6.1-1.el7    leveldb.x86_64 0:1.12.0-11.el7                libbabeltrace.x86_64 0:1.2.4-3.el7        libcephfs2.x86_64 2:14.2.1-0.el7      libibverbs.x86_64 0:17.2-3.el7     
  liboath.x86_64 0:2.6.2-1.el7        librabbitmq.x86_64 0:0.8.0-2.el7        librados2.x86_64 2:14.2.1-0.el7               libradosstriper1.x86_64 2:14.2.1-0.el7    librbd1.x86_64 2:14.2.1-0.el7         librdmacm.x86_64 0:17.2-3.el7      
  librgw2.x86_64 2:14.2.1-0.el7       lttng-ust.x86_64 0:2.4.1-4.el7          python-ceph-argparse.x86_64 2:14.2.1-0.el7    python-cephfs.x86_64 2:14.2.1-0.el7       python-rados.x86_64 2:14.2.1-0.el7    python-rbd.x86_64 2:14.2.1-0.el7   
  python-rgw.x86_64 2:14.2.1-0.el7    rdma-core.x86_64 0:17.2-3.el7           userspace-rcu.x86_64 0:0.7.16-1.el7          

Complete!

</code>

====== On the mon server ======
Create a user for the cephfs:
<code bash>
ceph auth get-or-create client.cephfs-${CLIENTNAME} mon 'allow r' > /etc/ceph/client.cephfs-${CLIENTNAME}.keyring
</code>


Modify the ACL for the user:
<code bash>
vim  /etc/ceph/client.cephfs-${CLIENTNAME}.keyring
</code>
Add the capabilities, for example if you want that the client can ''rw'' on ''/'':
<code bash>
    caps mds = "allow r path=/, allow rw path=/"
    caps mon = "allow r"
    caps osd = "allow class-read object_prefix rbd_children, allow rw pool=${DATAPOOL}"
</code>

You must copy the ''key'', is the "password" for the client mounting the cephfs:
<code bash>
[client.cephfs-ftp]
        key = <THIS_IS_WHAT_YOU_HAVE_TO_COPY_-_INCLUDING-==_AT_THE_END>
    caps mds = "allow r path=/, allow rw path=/ftp"
    caps mon = "allow r"
    caps osd = "allow class-read object_prefix rbd_children, allow rw pool=cephfs_data-ftp"
</code>

===== back to the client =====

Write the copied ''key'' from the ''mon'' server to a secure location:
<code bash>
echo ${THEKEY} > /etc/ceph/client.secret
</code>

And mount the fs:
<code bash>
mkdir /mnt/cephfs
mount -t ceph monserver-001.ciberterminal.net:6789,monserver-002.ciberterminal.net:6789,monserver-003.ciberterminal.net:6789,monserver-004.ciberterminal.net:6789:/ /mnt/cephfs -o name=cephfs-${CLIENTNAME},secretfile=/etc/ceph/client.secret
</code>



Add the fs to the ''fstab'':
<code bash>
monserver-001.ciberterminal.net:6789,monserver-002.ciberterminal.net:6789,monserver-003.ciberterminal.net:6789,monserver-004.ciberterminal.net:6789:/ /mnt/cephfs ceph name=cephfs-${CLIENTNAME},secretfile=/etc/ceph/client.secret 0 0
</code>