User:Tapas.mishra

How to use Xen to host application servers which will have application servers on it. Complete Guide for Newbies.

To have four application servers which will be running on top of a Xen Server. These servers are having Debian Lenny as their operating system. These four servers will be hosting the applications which are critical to the functioning of organization.We are going to demonstrate how this is possible using Xen and Debian Linux distribution and Apache2.

Objective

To have four application servers which will be running on top of a Xen Server. These servers are having Debian Lenny as their operating system. These four servers will be hosting the applications which are critical to the functioning of organization.We are going to demonstrate how this is possible using Xen and Debian Linux distribution and Apache2.

System Description: Dell PowerEdge R710,Debian Lenny (amd64) bit, 500GB Hard disk, 8GB RAM, Quad Core Processor.

Key Terms:Virtualization,Reverse Proxy on Apache2,Xen,Running many servers on same page.

Who should read this document:

People/Organizations who want to have different servers for each application which will be dedicated to those applications without buying a separate hardware for these servers.System administrators who are new to Xen on Linux.Any one who is interested to test virtualization for their applications.Some one who wants to understand how things work for their knowledge. Before you go to read this document familiarize yourself with DNS,Gateway and some basic networking stuff. That will be fine. Those who want to go a level higher they should learn use of IPTABLES on Linux.You do not need to worry about IPABLES if you are not clear with that part.

Table of Contents

* 1 What is virtualization * 2 Test Infrastructure and Network * 3 Section A Preparing the base system Dom0 * 4 Step B Installing Xen on Debian Lenny * 5 Step C: Creating LVM based DomU Images on Xen for Debian Lenny * 6 Step D:Using the created Virtual Machines i.e. Installed Guest Operating System * 7 Step E: Knowing the configuration files * 8 Step F:Setting up of Networking on Xenserver and DomU's   * 9 Reverse proxy using Apache2 on Dom0 and DomU's    * 10 Check points and possible errors * 11 References

What is virtualization
. Virtualization is the name given to a concept where in you run different many different Operating Systems as Guest. . Operating systems using a layer which is known as hypervisor. In this document we have used Xen.These Guest Operating Systems can be Windows,Linux,Solaris or any other operating sytems of your choice and you can always have a different operating system as Host OS. There are commercial applications such as vmware which can be used in place of Xen. KVM,OpenVZ are among the other applications which can be used in place of Xen. Which hypervisor you want to use is totally upon your choice.Discussion of Virtualization is beyond the scope of  this document and this small page about Virtualization should give you a basic insight about Virtualization.If you  want to read more about Virtualization you should search internet.You can read books about Virtualization here are a  few to name. . .  . Amazon Link . http://www.amazon.com/Running-Xen-Hands-Guide-Virtualization/dp/0132349663 . Running Xen: A Hands-On Guide to the Art of Virtualization by Jeanna N. Matthews; Eli M. Dow; Todd Deshane; Wenjin Hu; Jeremy Bongio; Patrick F.  Wilbur; Brendan Johnsonby Jeanna N. Matthews; Eli M. Dow; Todd Deshane; Wenjin Hu; Jeremy Bongio; Patrick F.   Wilbur;Brendan Johnson . . Another is : Xen Virtualization . http://www.amazon.com/Xen-Virtualization-practical-supporting-hypervisor/dp/1847192483  By:PrabhakarChaganti . If you are already aware of basics of Virtualization then you may skip this section and move ahead to next section. .

Test Infrastructure and Network
. We are going to install Debian on Dell PowerEdge R710 Server. We have a network whose configuration is as follows . Dom0 with IP      192.168.0.100 (This will be our Xen Server) DomU1 with IP     192.168.0.11 DomU2 with IP     192.18.0.12 DomU3 with IP     192.168.0.13 DomU4 with IP     192.168.0.14 . Subnet for all of the above is 255.255.255.0 . Gateway to network is 192.168.0.1

Section A: Preparing the base system Dom0
. 1)  Get a copy of Debian amd64 version. This can be obtained from here . http://cdimage.debian.org/debian-cd/5.0.4/amd64/iso-cd/  download   debian-504-amd64-CD-1.iso . 2)A firmware upgrade was needed at the time of writing this guide to make LAN card work. . Firmware update which made R710 lan card work . http://packages.debian.org/lenny/all/firmware-bnx2/download http://lists.us.dell.com/pipermail/linux-poweredge/2009-April/039119.html . 3) Install Debian Lenny (amd64) on Dell Poweredge R710. . Choose 10GB partition to install.We choose 10GB to install base system on it. Choose 30GB for swap space.These specifications are not strictly necessary you can have your own choices depending on your use. . Rest of the 460GB leave empty creating 4 different partitions. Size of partitions remaining is 160,100,100,100 GB partitions. You can leave them as it is or can create a LVM group which consist of all the  above partition in same volume group.After the base system Debian Lenny has been installed and you have upgraded  firmware and booted the base system,you should update sources.list file.In case you run through some errors in your netwok or LAN before proceeding to the  next section go through the errors section. . Do following

xenserver#cat >> /etc/apt/sources.list deb http://ftp.us.debian.org/debian/ lenny main deb-src http://ftp.us.debian.org/debian/ lenny main deb http://security.debian.org/ lenny/updates main contrib deb-src http://security.debian.org/ lenny/updates main contrib deb http://ftp.us.debian.org/debian/ lenny non-free deb http://ftp.us.debian.org/debian/ lenny contrib

. and press enter.Your /etc/apt/sources.list should look like this deb http://ftp.us.debian.org/debian/ lenny main deb-src http://ftp.us.debian.org/debian/ lenny main deb http://security.debian.org/ lenny/updates main contrib deb-src http://security.debian.org/ lenny/updates main contrib deb http://ftp.us.debian.org/debian/ lenny non-free deb http://ftp.us.debian.org/debian/ lenny contrib

. Note: After you have installed DomU's the above mirrors can also be used onDomU's . 4) Update your network settings i.e. IP, gateway,DNS,netmask etc. . vi /etc/network/interfaces

#allow-hotplug eth0 auto eth0 iface eth0 inet static address 192.168.0.100 netmask 255.255.255.0 network 192.168.0.0 broadcast 192.168.0.255 gateway 192.168.0.1
 * 1) The loopback network interface auto lo iface lo inet loopback
 * 2) The primary network interface
 * 1) iface eth0 inet dhcp

. Then restart your network: /etc/init.d/networking restart

Then edit /etc/hosts.

. vi /etc/hosts

127.0.0.1      localhost.localdomain           localhost 192.168.0.100 xenserver.lan.ernet.in           xenserver

. Modify the settings for your network replace the domain names with what ever is applicable for you. . Edit /etc/resolv.conf and add some nameservers:

vi /etc/resolv.conf

nameserver 192.168.1.24 nameserver 192.168.1.25

where 192.168.1.24 and 192.168.1.25 are DNS servers.

They should contain entries which are relevant for you.

. 5) apt-get update . . 6)Do apt-get install ssh vim-full

Step B Installing Xen on Debian Lenny
. There are two ways to install Xen. . a) Compile Xen from sources.This one you can read here . ["http://mightydreams.blogspot.com/2010/04/compiling-xen-from-sources-on-debian.html"]  . b) Install a precompiled binary for your architecture. . We have chosen the second approach.

Step 1) Installing Xen

apt-get install xen-hypervisor-3.2-1-amd64    xen-linux-system-2.6.26-2-xen-amd64 xen-utils-3.2-1 xenstore-utils   xenwatch     xen-shell    xen-tools

To check what these packages contain you can check

. http://packages.debian.org/lenny/xen-hypervisor-3.2-1-amd64 . http://packages.debian.org/lenny/xen-linux-system-2.6.26-2-xen-amd64 . http://packages.debian.org/lenny/xen-utils-3.2-1 . http://packages.debian.org/lenny/xenstore-utils . http://packages.debian.org/lenny/xenwatch . http://packages.debian.org/lenny/xen-shell . http://packages.debian.org/lenny/xen-tools . While installing above note the messages some of them download the xen-utils package so you need not to re . install them again. Before you proceed to check that which package is installed you can do so by dpkg -s

Step2) open /etc/xen/xend-config.sxp

vi /etc/xen/xend-config.sxp

. We have used a bridged network setup if you want to use NAT or routed setup based on IPTABLES then there will be a different setup. . Step 3) uncomment the line

(network-script network-bridge)

. and comment out the line

. Also make sure that the line (vif-script vif-bridge) is enabled: . The above file should appear some thing like this . Then reboot the . system: reboot . and check . check uname -r and your new Xen kernel should show up: .   . you can do an xm info an output like some thing as follows should come up

. = Step C: Creating LVM based DomU Images on Xen for Debian Lenny = . Before we go on to proceed with LVM based images we will check for the LVM setup.You can skip this section if you have already set a LVM based setup and go to Creation ofLVM based images. . Check your hard disk structure

. First you need to update some packages. . The above system configuration has 4 hard disks partitions .

. Check point if you did not made 4 partitions while installing you need to do some home work. . You can run pvdisplay to learn about the current state of your physical volumes. .  . Now we will create our volume group virtualization and add /dev/sda6 - /dev/sda7-    /dev/sda8  /dev/sda9

. '''      You do not need to create LVM here. LVM will be created by the xen-create-image ''' . it will create a swap and root partition which will be an LVM image.

Creation of LVM based images
. . Here mirror is pointing to a debian repository which is used for installation.If you use a  loop mounted CDROM then you can replace it with the location of mounted directory. I have mentioned both the methods below you can choose any. Do not forget to chage serial_device = hvc0 and disk_device =xvda as shown above. . What ever you write above will be replaced by anything that you will specify on command    line. .  Most Imp:   Specify IP of gateway as the IP of your Dom0 We do not need to make or compile a DomU kernel.We can use default debian kernels on   the installation media. Default installation OS type above in file was etch where as we are using lenny so replace it by that. Size above 160GB can vary depending upon availability of  hard disk space you have. . The dir line specifies where the virtual machine images will be stored. ' dist specifies the distribution to be installed in the virtual machines (Debian Lenny) (there's a comment in the file . that explains what distributions are currently supported). The passwd = 1 line makes that you can specify a root password when you create a new   guest domain. Make sure that you uncomment the lvm line and fill in the name of your     volume group (virtualization is the volumegroup here). At the same time make sure that the dir line is commented out. . It is very important that you add the line serial_device = hvc0 (not xvc0 or tty1)otherwise  the console of your . virtual machines might not work properly. Method 1 (If you do not want to go for internet one you may skip it and go to Method2) Using internet based repository . What ever you specify on the command line here will over write the . /etc/xen-tools/xen-tools.conf You may specify a different internet mirror above. Note –role-udev above do not miss it. Method 2 (You can skip it and go to section image ) Using CD as your local repository . http://cdimage.debian.org/debian-cd/5.0.4/amd64/iso-cd/ . If you have the CD image burnt on to a CD do the following insert Debian CD in CDROM

. After a few minutes you will get a message saying that the CD has been copied.Same . command will work for DVD also.You need to check your DVD rom in your /dev. . Now make a directory named local in your home directory i.e.(the name local is not strictly required you may choose  . what ever you wish it) . You can navigate to directory local and see the contents of CDROM in local directory. . Image . The xen-create-image command will now create the domu1 virtual machine for us. This will take a few minutes. The output should be similar to this one:

. There should now be a domu1 configuration file - /etc/xen/domu1.cfg. If you have a dual-core or quad-core CPU and want the virtual machine to use all CPU   cores, please add the line vcpus = . '2' or vcpus = '4' to the configuration file If you are wondering that we have not created Logical Volumes then just notice the outputs   of above commands which have . created separate root and swap partitions.You can confirm it  by command lvdisplay . Now you have one DomU1 up same process you can apply to create other instances. =Step D:Using the created Virtual Machines i.e. Installed Guest Operating Systems= . To be able to use the newly created guest i.e. Virtual machine whose name is domu . xm create /etc/xen/domu1.cfg and then run xenserver#xm console domu1 The output should look like this: To shut down domu1, do this: xm shutdown domu1 . For more commands you can use xm help. . Now suppose you have 4 different DomU's installed If you want domu1 to start automatically at the next boot of the system, then do this: ln -s /etc/xen/domu1.cfg /etc/xen/auto

Step E: Knowing the configuration files
. A sample configuration file for domU looks like this

. A list of virtual machines that were created with the xen-create-image command can be found by xen-list-images =      Step F:Setting up of Networking on Xenserver and DomU's=

. When you install 4 DomU's with Ips on them it is as good as bringing 4 hosts on network. . In our case . We have following

. So our network structure is some thing like that . DomU1           DomU2                DomU3               DomU4 192.168.0.11   192.168.0.12        192.168.0.13          192.168.0.14  . a                   b                  c                      d   . Dom0 . 192.168.0.100                                     [XenServer] . IP Forwarding Enabled on Dom0 . Figure1 . We have to enable Ip forwarding.For the hosts to appear on network. . xenserver:~# echo 1 > /proc/sys/net/ipv4/ip_forward . also in /etc/sysctl.conf uncomment net.ipv4.ip_forward=1 . Now your hosts should be appearing on network.Do not have any other host have the same IP as DomU's otherwise IP collision will occur.So suppose you have a machine C the structure is now like this . Note : Inspite of the fact that that a,b,c,d are DomU's they are available on network as  . actual machines. This is what you should have achieved by now. . Figure 2 When you login to any of the DomU you should set the gateway as Dom0. . DNS can be pointing to actual DNS of your network since with IP Forwarding enabled the DomU's will be able to access the proxy and rest of the network. . Now since you have enabled IP Forwarding so you should be able to ping from any other machine to you DomU's to make sure the things are working as expected you ping from C to a you should get a reply. We are at C and we can check . ping  192.168.0.11  some out put like following will come

. So that means that our host that is DomU1 is accessible from C and in turn from the rest of    the network. Similary ssh and other things should work. If it does not happen then check if you have       enabled ip forwarding. You do NOT need to enable IP FORWARDING on DomU's  . IP Forwarding should be enabled only on Dom0. Xen has a default network managment which will create a bridge whose name will be same as your interface name.If you want to read more about it look at the following  Wiki page. http://wiki.xensource.com/xenwiki/XenNetworking

. We have default ethernet as eth2 ,it can be different on your network settings. . If some thing on your LAN card does not work check  /etc/xen/xend-config.sxp Here is a sample note netdev=eth2 in below output These are my entries . vi /etc/xen/xend-config.sxp

. If   (network-script 'network-bridge netdev=eth2') is commented or missing some thing then uncomment and put the name netdev= as above. . and there is one more entry you should check that is (vif-script vif-bridge) vi /etc/xen/xend-config.sxp

. So by now your all 4 hosts must be up and running on your network and you should be able to access these DomU's from  any other machine on your network.

Section 1:  Setting up of Apache2 on Dom0 and DomU
. Before you proceed to this section and do as mentioned make sure that your guest Operating Systems or DomU's are able  to access your LAN. . We are doing this with Dom0 which has a public IP or say internet access without any proxy. . To make apt-get work with proxy . mention following lines in .bashrc on individual DomU's. . Now its time for work aptitude install mysql-server mysql-client You will be asked to provide a password for the MySQL root user - this  password is valid for the user root@localhost New password for the MySQL "root" user:   <-- yourrootsqlpassword Repeat password for the MySQL "root" user: <-- yourrootsqlpassword . Apache2 is available as a Debian package, therefore we can install it like this: Now direct your browser to http://192.168.0.100, and you should see the Apache2 placeholder page (It works!): Repeat all these steps  for respective DomU's  so that you install apache2 on all DomU's and Dom0. . From your LAN you can check http://IP of Dom0/ . I created Dom0 on IP 192.168.0.100 Now open a browser and go to following url http://192.168.0.100 . you should get a message It Works!! and same for all DomU's and Dom0 check in browser and go to following urls . http://IP of DomU1/            http://192.168.0.11/ http://IP of DomU2/             http://192.168.0.12/ http://IP of DomU3/             http://192.168.0.13/ http://IP of DomU4/             http://192.168.0.14/ you should get a message '''It Works!! ''' Replace the IP of DomU with what ever you have in your network. =Section 2: Setting up for access to internet for application servers (Reverse Proxy)= . Now go to Dom0 and open your command prompt

. Now reload Apache: /etc/init.d/apache2 force-reload vi /etc/apache2/apache2.conf

. Right before this section, we add the following lines:

. Now you should be able to access DomUs from Internet as well.If you have a public IP on      Dom0. . Suppose you have a domain name www.myxenserver.com So you should be able to access . http://www.myxenserver.com/domu1 . http://www.myxenserver.com/domu2 . http://www.myxenserver.com/domu3 . http://www.myxenserver.com/domu4 . . .        Then restart Apache: /etc/init.d/apache2 restart

Check points and possible errors
. The first time I booted the system Lan Card driver was not recognized. I got a error firmware: requesting bnx2-09-4.0.5.fw it is available on following page . http://packages.debian.org/lenny/all/firmware-bnx2/download Reboot then. If you run into any sort of problem on network check if your IP is conflicting with any other Ip on network ,ping the gateway,DNS. . . Open a terminal and type . ping  . ping  . May be network card does not seems to be up then check as root . . ifconfig -a . the lan card which you configured might be not the default eth0 it might be eth1 or eth2 or eth3 like that. You need to check the LAN cable in your rack. . . Try different permutations to see that when you ping some machine on your network you . get a response.I was running out of space when I was creating DomU's. It may happen with you also. . Before you choose to install separate OS as DomU's calculte the amount of RAM they will              be consuming and swap space that you will be allocating them. When Dom's are running they require a certain amount of memory to be allocated to them suppose you have 8 GB RAM and you decide to give 2GB Ram to each of the virtual host you            create i.e. DomU then you wont be left with memory for Dom0. We planned to give 875 Mb memory to the Dom0 so we were left with 8192-875=7317Mb of Ram which we planned to give to the DomU's.If you notice the output of xm list you can see it

. Where domu1,domu2,domu3,domu4 are different dom's running.