User Tools

Site Tools


linux:high_availability_virtualization_cluster

Differences

This shows you the differences between two versions of the page.


Previous revision
linux:high_availability_virtualization_cluster [2022/02/11 11:36] (current) – external edit 127.0.0.1
Line 1: Line 1:
 +====== [HOWTO] Linux KVM cluster ======
  
 +====== Description ======
 +Instructions on how to deploy a high availability KVM cluster based on CentOS 6.x
 +
 +
 +====== Instructions ======
 +===== RHEL cluster =====
 +  * Deploy Centos (minimal)
 +  * Enjoy with basic networking.
 +  * Disable SELINUX
 +  * Install EPEL meta-pkg:
 +<code bash>
 +rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
 +</code>
 +  * Install basic pkgs (for me):
 +<code bash>yum -y install vim tmux lsof strace
 +</code>
 +  * Update:
 +<code bash>yum update
 +</code>
 +  * Enjoy with multipathd+storage 8-)
 +  * Install the clusterware suite:
 +<code bash>yum groupinstall "High Availability" "Resilient Storage"
 +</code>
 +  * In the "master" node, install the admin suite:
 +<code bash>yum groupinstall "High Availability Management"
 +</code>
 +  * Change the password for "ricci" user (all nodes):
 +<code bash>passwd ricci</code>
 +  * Configure services:
 +<code bash>
 +chkconfig ricci on
 +chkconfig luci on
 +chkconfig cman on
 +chkconfig modclusterd on
 +</code>
 +  * Start services:
 +<code bash>
 +service ricci start
 +service cman start
 +</code>
 +  * Start luci on admin node:
 +<code bash>service luci start</code>
 +If luci refuses to start, read the [[high_availability_virtualization_cluster#Troubleshoot|troubleshoot]] section.
 +  * Disable NetworkManager:
 +<code bash>
 +chkconfig NetworkManager off && service NetworkManager stop
 +</code>
 +  * Setup at least one bridge, for example having 4 interfaces: eth0+eth1=bonding, eth2+eth3=bridge:
 +    * ifcfg-eth[01]
 +<code bash>
 +DEVICE=eth[01]
 +TYPE=Ethernet
 +ONBOOT=yes
 +NM_CONTROLLED=no
 +BOOTPROTO=none
 +MASTER=bond0
 +SLAVE=yes
 +</code>
 +    * ifcfg-eth[23]
 +<code bash>
 +DEVICE=eth[23]
 +ONBOOT=yes
 +NM_CONTROLLED=no
 +BOOTPROTO=none
 +BRIDGE=br0
 +</code>
 +    * ifcfg-br0
 +<code bash>
 +DEVICE=br0
 +TYPE=Bridge
 +ONBOOT=yes
 +BOOTPROTO=none
 +USERCTL=no
 +</code>
 +    * ifcfg-bond0
 +<code bash>
 +DEVICE=bond0
 +ONBOOT=yes
 +BOOTPROTO=static
 +USERCTL=no
 +NETWORK=10.54.0.0
 +IPADDR=10.54.0.20
 +NETMASK=255.255.255.0
 +BROADCAST=10.52.0.255
 +GATEWAY=10.54.0.1
 +</code>
 +  * Enable forwarding on bridges on iptables:
 +<code bash>
 +iptables -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT
 +</code>
 +  Or disable iptables (if you have a good firewall):
 +<code bash>
 +service iptables stop
 +chkconfig iptables off
 +</code>
 +  * Enable forwarding on the kernel:
 +<code bash>
 +inet.ipv4.ip_forward = 1
 +sysctl -p /etc/sysctl.conf
 +</code>
 +  * Access the luci UI or Conga formatively (the url is displayed after staring luci, something like https://admin_node:8084/) and create the cluster.
 +  * Define some Fence method for each node.
 +  * I suppose you've exported some volume from your storage system to all the servers and you've setted up multipathd (or whatever) so you can see the volume under ''/dev/mapper''.
 +  * Create the partition on the disk:
 +<code bash>
 +parted /dev/mapper/SHAREDDISK
 +(parted) mklabel gpt
 +(parted) mkpart primary ext2 0 9999999G
 +(parted) set 1 lvm on
 +(parted) quit
 +</code>
 +  * Create the lvm on the new partition:
 +<code bash>pvcreate /dev/mapper/SHAREDDISKp1
 +vgcreate --clustered y SHAREDVG /dev/mapper/SHAREDDISKp1
 +</code>
 +  * On the rest of the nodes (I didn't needed it, but Its safer to run it):
 +<code bash>
 +partprobe; vgscan
 +</code>
 +  * Create the LVM volume:
 +<code bash>lv create -L 9999999G -n SHARED_LV SHAREDVG
 +</code>
 +  * And create the GFS2:
 +<code bash> mkfs.gfs2 -p lock_dlm -t CLUSTERNAME:SHAREDVOLUME -j 4 /dev/mapper/SHAREDVG-SHARED_LV
 +</code>
 +  * Add the new filesystem to fstab (**MANDATORY**) on all the nodes:
 +<code bash>
 +# GFS
 +/dev/mapper/SHAREDVG-SHARED_LV            /mnt/shared_storage         gfs2    noatime         0 0
 +# GFS
 +</code>
 +  * Mount it
 +
 +
 +
 +===== KVM =====
 +  * Install the dependencies:
 +<code bash> yum groupinstall "Virtualization Platform" "Virtualization Tools" Virtualization "Virtualization Client"
 +</code>
 +  * More deps for virt-manager:
 +<code bash>
 +yum install dejavu-lgc-sans-fonts
 +</code>
 +
 +
 +
 +
 +<code bash>
 +</code>
 +
 +
 +
 +
 +====== Troubleshoot ======
 +===== Luci =====
 +If you get an error like this:
 +<code>
 +Unable to create the luci base configuration file (`/var/lib/luci/etc/luci.ini').
 +</code>
 +
 +You can try to:
 +<code bash>
 +rpm -e --nodeps python-webob-0.9.6.1-3.el6.noarch
 +easy_install WebOb==1.0.8
 +</code>
 +**This should be enough**, but if you're still having problems running luci, follow with that instructions:
 +Edit that file:
 +<code bash>
 +vi /usr/lib/python2.6/site-packages/pylons/decorators/__init__.py
 +</code>
 +And comment/remove the line (line ~20):
 +<code bash>
 +from webob import UnicodeMultiDict
 +</code>
 +
 +That is.
 +
 +[[http://scientificlinuxforum.org/index.php?s=e8f367a66e9529a1e2acd0b4b1a765f7&amp;showtopic=939&view=findpost&p=7214|thanks to dmabry]] and [[https://groups.google.com/d/msg/turbogears/PtcScbOX-C0/G1WwKnIn04MJ|Michael Pedersen]] combining their post I were able to run Luci.