Using LVS - notes

Using Linux Virtual Server modules for high-availability systems,

Heartbeat
---------

Heartbeat (v2) lives in /etc/ha.d (or /etc/heartbeat - symbolic link in Debian)

Important files in this directory include:-

ha.cf - general configuration file for heartbeat
haresources - resources we want to maintain AND the server we would prefer to run them on in Active/Standby
authkeys - authentication keys for encrypted messaging between heartbeat nodes

sample files are found in /usr/share/doc/heartbeat-2 in Debian

(resource.d) - Directory: Conatains special scripts to handle failover situations
(conf) - Directory: Contains configuration files which scripts in resource.d use in failover situations

And now for some detailed info on the files:-

ha.cf
-----

Important things to configure here are:-

logfacility - sort your heartbeat logging out (if any)

keepalive
deadtime
initdead - important timeout limits to keep when things die

udpport - need to have one to communicate between machines with

bcast - which ethernet ports heartbeat will communicate out from (generally more than one to ensure that the other machine has died)

auto_failback - if a dead machine reappears, shall it takeover from the failover machine?

node - what are the node names in your cluster (use uname to determine this)

crm - turn it off unless you like to trawl through xml configuration files

haresources
-----------

resources here are defined as "servername resource resource ...."

The servername must be the matching node-name (from ha.cf)
Resources are scriptnames. The scripts are either found in /etc/init.d or in the /etc/ha.d/resource.d directory

Arguments to the scripts are delimited with "::"
Without arguments, scripts are treated with stop/start as dictated by the failover

haresources has to be the same on both machines! Otherwise the system goes crazy

The ipvs module
---------------

This is a module compiled into the Linux kernel. Usually in there by default, HOWEVER the number of simultaneous connections has to be set in the kernel. This means that recompiling the kernel is necessary to support more than 4096 (default Debian kernel setting) simultaneous conenctions.

Once it is in there, and activated, ldirectord is run under heartbeat to manage it
(although you can manipulate it from the command line too) 

So the complete stack is IPVS module --> ldirectord --> heartbeat

Also, a sysctl needs to be modified into order for the module to work properly.

in /etc/sysctl.conf, set net.ipv4.ip_forwarding=1


ldirectord
-----------

This gets installed in Debian by throwing a link to the daemon in /etc/ha.d/resource.d
A suitable failover script then can be placed in the "conf" directory and called from there in haresources
with a single argument.

ldirectord is like a "front end" for the ipvs kernel module. It is here that you can set virtual ips and services. You can see the results of this module with ipvsadm

you can name the ldirectord configuration script however you like, it will be called from heartbeat's haresources file anyway.

The ldirectord configuration contains a lot of options. All options are considered global unless
preceded by a "virtual" option (which specifies the virutal IP and port to be used) 

checktimeout - seconds for other server timeout before taking action

checkinterval - seconds to check status

autoreload - if the configuration should be continuously checked for updates

quiescent - best to leave this as "yes", which means that timed out servers get weight 0 and no new connections (this gives old connections somewhere to go at least). No quiescence means that the lvs table entry abrubtly cuts out

virtual=: - sets up the virtual IP
	(and from this point on anything under this setting is for this virtual IP)

real=: - list of real servers here and how they are to behave

service - the type of service to monitor (a number of services are possible. eg. http, https, mysql)

scheduler - this matches the ipvsadm matching scheduler type. By default you will get wrr (weighted round robin). Some other popular choices are sh (source hashing) and wlc (weighted least connection)

protocol - if the virtual handles a TCP port, then this should be TCP (or UDP if it's a UDP port)

checktype - method of connecting to other machine to test for availability. "connect" starts up a TCP session and is pretty fast and good enough. There are also ping and negotiation tests available, but cause more overhead.

emailalert - include an email address (or simply "root") to get notifications of failover

emailalertfreq - how many seconds between alerts.

DRBD
-----

DRBD is currently an unofficial kernal patchset for Linux (They are hoping to get this into the mainline kernel at somepoint)