Differences

This shows you the differences between two versions of the page.

--- high_availability_virtualization_cluster [2013/08/29 14:31] – created dodger
+++ linux:high_availability_virtualization_cluster [2022/02/11 11:36] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
+====== [HOWTO] Linux KVM cluster ======
 ====== Description ======
 Instructions on how to deploy a high availability KVM cluster based on CentOS 6.x
@@ Line 4: / Line 6: @@
 ====== Instructions ======
-Deploy Centos (minimal?)
+===== RHEL cluster =====
+  * Deploy Centos (minimal)
+  * Enjoy with basic networking.
+  * Disable SELINUX
+  * Install EPEL meta-pkg:
+<code bash>
+rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
+</code>
+  * Install basic pkgs (for me):
+<code bash>yum -y install vim tmux lsof strace
+</code>
+  * Update:
+<code bash>yum update
+</code>
+  * Enjoy with multipathd+storage 8-)
+  * Install the clusterware suite:
+<code bash>yum groupinstall "High Availability" "Resilient Storage"
+</code>
+  * In the "master" node, install the admin suite:
+<code bash>yum groupinstall "High Availability Management"
+</code>
+  * Change the password for "ricci" user (all nodes):
+<code bash>passwd ricci</code>
+  * Configure services:
+<code bash>
+chkconfig ricci on
+chkconfig luci on
+chkconfig cman on
+chkconfig modclusterd on
+</code>
+  * Start services:
+<code bash>
+service ricci start
+service cman start
+</code>
+  * Start luci on admin node:
+<code bash>service luci start</code>
+If luci refuses to start, read the [[high_availability_virtualization_cluster#Troubleshoot|troubleshoot]] section.
+  * Disable NetworkManager:
+<code bash>
+chkconfig NetworkManager off && service NetworkManager stop
+</code>
+  * Setup at least one bridge, for example having 4 interfaces: eth0+eth1=bonding, eth2+eth3=bridge:
+    * ifcfg-eth[01]
+<code bash>
+DEVICE=eth[01]
+TYPE=Ethernet
+ONBOOT=yes
+NM_CONTROLLED=no
+BOOTPROTO=none
+MASTER=bond0
+SLAVE=yes
+</code>
+    * ifcfg-eth[23]
+<code bash>
+DEVICE=eth[23]
+ONBOOT=yes
+NM_CONTROLLED=no
+BOOTPROTO=none
+BRIDGE=br0
+</code>
+    * ifcfg-br0
+<code bash>
+DEVICE=br0
+TYPE=Bridge
+ONBOOT=yes
+BOOTPROTO=none
+USERCTL=no
+</code>
+    * ifcfg-bond0
+<code bash>
+DEVICE=bond0
+ONBOOT=yes
+BOOTPROTO=static
+USERCTL=no
+NETWORK=10.54.0.0
+IPADDR=10.54.0.20
+NETMASK=255.255.255.0
+BROADCAST=10.52.0.255
+GATEWAY=10.54.0.1
+</code>
+  * Enable forwarding on bridges on iptables:
+<code bash>
+iptables -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT
+</code>
+  Or disable iptables (if you have a good firewall):
+<code bash>
+service iptables stop
+chkconfig iptables off
+</code>
+  * Enable forwarding on the kernel:
+<code bash>
+inet.ipv4.ip_forward = 1
+sysctl -p /etc/sysctl.conf
+</code>
+  * Access the luci UI or Conga formatively (the url is displayed after staring luci, something like https://admin_node:8084/) and create the cluster.
+  * Define some Fence method for each node.
+  * I suppose you've exported some volume from your storage system to all the servers and you've setted up multipathd (or whatever) so you can see the volume under ''/dev/mapper''.
+  * Create the partition on the disk:
+<code bash>
+parted /dev/mapper/SHAREDDISK
+(parted) mklabel gpt
+(parted) mkpart primary ext2 0 9999999G
+(parted) set 1 lvm on
+(parted) quit
+</code>
+  * Create the lvm on the new partition:
+<code bash>pvcreate /dev/mapper/SHAREDDISKp1
+vgcreate --clustered y SHAREDVG /dev/mapper/SHAREDDISKp1
+</code>
+  * On the rest of the nodes (I didn't needed it, but Its safer to run it):
+<code bash>
+partprobe; vgscan
+</code>
+  * Create the LVM volume:
+<code bash>lv create -L 9999999G -n SHARED_LV SHAREDVG
+</code>
+  * And create the GFS2:
+<code bash> mkfs.gfs2 -p lock_dlm -t CLUSTERNAME:SHAREDVOLUME -j 4 /dev/mapper/SHAREDVG-SHARED_LV
+</code>
+  * Add the new filesystem to fstab (**MANDATORY**) on all the nodes:
+<code bash>
+# GFS
+/dev/mapper/SHAREDVG-SHARED_LV            /mnt/shared_storage         gfs2    noatime         0 0
+# GFS
+</code>
+  * Mount it
+===== KVM =====
+  * Install the dependencies:
+<code bash> yum groupinstall "Virtualization Platform" "Virtualization Tools" Virtualization "Virtualization Client"
+</code>
+  * More deps for virt-manager:
+<code bash>
+yum install dejavu-lgc-sans-fonts
+</code>
+<code bash>
+</code>
+====== Troubleshoot ======
+===== Luci =====
+If you get an error like this:
+<code>
+Unable to create the luci base configuration file (`/var/lib/luci/etc/luci.ini').
+</code>
+You can try to:
+<code bash>
+rpm -e --nodeps python-webob-0.9.6.1-3.el6.noarch
+easy_install WebOb==1.0.8
+</code>
+**This should be enough**, but if you're still having problems running luci, follow with that instructions:
+Edit that file:
+<code bash>
+vi /usr/lib/python2.6/site-packages/pylons/decorators/__init__.py
+</code>
+And comment/remove the line (line ~20):
+<code bash>
+from webob import UnicodeMultiDict
+</code>
+That is.
+[[http://scientificlinuxforum.org/index.php?s=e8f367a66e9529a1e2acd0b4b1a765f7&amp;showtopic=939&view=findpost&p=7214|thanks to dmabry]] and [[https://groups.google.com/d/msg/turbogears/PtcScbOX-C0/G1WwKnIn04MJ|Michael Pedersen]] combining their post I were able to run Luci.