Upgrading Citrix XenServer 6.x to 7

I know it’s been a little bit, I intended to blog about this a month ago but life and work sometimes get busy. Nonetheless here goes!  I know it’s been a little bit, I intended to blog about this a month ago but life and work sometimes get busy. Nonetheless here goes!

 

Step 1: Install Citrix XenCenter 7

Before you begin the upgrade process it’s crucial to install the new version of Citrix XenCenter as you will not be able to connect to or manage hosts post upgrade with the old client, however the new XenCenter client is backwards compatible. Both the XenCenter installer and XenServer ISO are available for download at the following link here. During the install of the client, the old XenCenter will be automatically removed and replaced by the new one.

 

Step 2: Start Pool Upgrade

If you are running a standalone XenServer host, the process is quite simple, you can skip the next paragraph as the steps will pertain to a multi-node Citrix XenServer pool.

When performing upgrades to a XenServer pool you will want to start by upgrading the pool master first. There are 2 options when performing the upgrade, you can choose to have the upgrade automatically move through the pool a host at a time, or you can choose to manually upgrade each host. For the purposes of this article I will cover the steps for the manual upgrade as most Xen admins will take the conservative approach that will give them the most control during the upgrade process. Citrix recommends backing up the pool database which can be accomplished by running xe pool-dump-database. You will want to live migrate or shutdown all running VMs on the given host you are about to upgrad.

Next we will choose the Tools menu and select Rolling Pool upgrade, and follow the steps in the wizard selecting the manual upgrade mode. You can then connect the ISO to the host via virtual media to the LOM/OOB interface (iDrac for Dell) or burn the ISO to disk for XenServer7 and insert the CD and connect to the host with a monitor and keyboard. Your server will reboot and you will need to choose the boot option to boot from disk or virtual media.

 

Step 3: Installing XenServer 7 on the Host

Upon booting from the disk you will be greeted by the screen shown below, press enter and continue.

boot from disk

You will be prompted to select your keymap, choose the relevant one for your language and keyboard type then press enter:

Screenshot from 2016-07-27 21-09-27

Select OK and press enter on the next screen to start the upgrade process:

Screenshot from 2016-07-27 21-09-43

Choose to accept the EULA and press enter:

Screenshot from 2016-07-27 21-10-13

Choose to upgrade from the existing XenServer and select OK:

Screenshot from 2016-07-27 21-10-25

Next you will receive a notice that the installer will collect a backup of your current installation, go ahead and continue:

Screenshot from 2016-07-27 21-10-49

Next we will need to choose the location of our installer, if you are using virtual media or burned a CD/DVD choose local media:

Screenshot from 2016-07-27 21-11-02

Next you will be prompted as to whether or not you would like to install any supplemental packs, if you have any that you are currently dependent on you may do so, if not choose no and proceed with the install (supplemental packs can also be installed after the fact)

Screenshot from 2016-07-27 21-11-21

Next we will be asked if we want to verify the installation source, you can safely skip this or if you’d like you can choose to verify (if you are using a method other than local install you should verify the install source):

Screenshot from 2016-07-27 21-11-35

Next the moment we’ve all been waiting for, choose install XenServer and go get a cup of coffee, this will take 10-15 minutes to complete.

Screenshot from 2016-07-27 21-11-50

Once this is complete your host has been fully upgraded and you should be able to see and manage it within XenCenter. If this is a standalone XenServer you are done upgrading hosts. If you are using a XenServer pool you will need to rinse and repeat this action across the other hosts in the pool.

Step 4: Update XenServer Tools

Once all of your hosts are upgraded you can now perform the XenServer Tools upgrade, this will greatly improve performance on Windows boxes, as Disk IO is not optimized until the new tools are installed and performance may be untenable and/or horrific until this is done. Please note for Windows upgrades it will take at least 2 sometimes 3 reboots for the upgrade to fully complete. For Linux systems you may need a reboot, however there are no performance issues with Linux guests running the same tools as the previous version of XenServer.

 

 

Citrix XenServer 7 Why Should I Upgrade?

Citrix XenServer has been my favorite Type 1 Hypervisor platform for the past year now for various reasons, mostly that it’s closer to VMWare than any other hypervisor and doesn’t come at the premium price tag of VMWare, and its control domain runs Centos and features a highly robust intuitive CLI. I’ve been watching XenServer 7 (codename Dundee) with much excitement from the time it went into public Alpha and I’m happy to announce it arrived as a full release on May 24th. Having fully upgraded my own Citrix pool at home and built one in the office at work I wanted to outline what’s new in XenServer 7 and some gotchas before my next blog post which will detail the upgrade process.

 

New Stuff!

We all like new and shiny, here’s a shortlist of new features and major improvements:

  • Improved graphics: it’s not surprise that we see performance improvements in this area as Citrix has been the market leader in VDI for sometime now.
  • Configuration Maximum increases: as with any new release of a hypervisor configuration maximums have increased, details below:
    • Hosts support up to 5TB RAM
    • Host supports up to 288 CPUs
    • Hosts can support up to  4096 Storaage Repositories
    • Guest VMs can now support up to 1.5TB of RAM
    • Guest VMs support up to 32 vCPUs
  • Docker Support: while this rolled out in 6.5 SP1 docker support now includes Docker containers running in Server 2016
  • Automated Windows VM XenTool Management
    • This was a major selling feature on upgrading however it should be noted that this only for new VMs created in XenServer 7 and does not include upgraded VMs.
    • Also of note is the process to upgrade XenTools is doggedly slow and the Guest VM runs terribly until the install finishes and there’s been a few reboot cycles. Once I/O is finally optimized performance is better than it was previously, however getting there can be painful and you may run into some issues with static IPs disappearing or the network adapter showing as a completely new adapter (which caused some headaches with Windows DHCP Server).
  • Support for SMB for VM disks (I haven’t personally used this feature as of yet)
  • SSH Console: We’ve all been familiar with the RDP prompt when using the console on a windows guest. The console now sports an Open SSH button for linux VMs that launches a Putty SSH session.
  • Dom0 Improvements: This is what we’ve all been waiting for. Dom0 is now substantially larger at 18GB, so no more worries about running out of space from logs or patching (at least not nearly as quickly)
    • The second notable improvement is the use of cgroups which help to keep a heavily loaded host from being unmanageable.
  • Server Health Check: The nagging prompt when first connecting to a new XenServer asking if you want to collect health reports and send them to Citrix. Unless you’re using a paid license this is a mostly useless feature, however I can see the benefit if you are paying for support.
  • THE OS: XenServer 7 now runs on CentOS 7. I was simply shocked when 6.5 came out to find that it was still running on the old trust CentOS 5. While I see this as a great step forward, be advised if you are using any scripts that rely heavily on sysvinit functions that this release now uses systemd and some tweaking to your scripts and automation tools may be required.

 

What Still Hasn’t Changed

  • Mounting iSO SRs from SMB v3.0 requires the NTLanManager local settings to be set to “Allow LM & NTLM and NTLMv2 if negotioated” on the Windows server hosting the share
  • Active Directory Integration still requires the above changes as well.

The Verdict

Other than the disappointment around the Windows XenTools mentioned above this was a very nice release and packs a lot of great new features and iterates successfully on what has made Citrix XenServer a great hypervisor. I honestly think if it were not for lack of marketing muscle, slow release cycles, and lack of partner integrations Citrix would be a bigger player in this space. Holding a Vmware VCA, Citrix CCA, and the MS Virtualization certifications I can say hands down that for 90% of my use cases Citrix XenServer is my go to hypervisor (unless I’m running 100% Linux, then KVM it is, or if a company has deep pockets and can afford the premium VMWare licensing). Stay tuned I’ll have an upgrade walkthrough coming in the next few weeks.

 

Setting Up Hyper-V Replication

Overview

Hyper-V Replication makes for a great built in feature to sync mission critical VMs to another host for warm standby. This feature allows for quick easy failover and takes minimal effort to setup. While this is intended to allow for failover and failback I have also utilized this replication methodology to migrate a 7 hosts worth of VMs from a datacenter on one side of the country to another. Generally speaking this process works best in a domain environment where both the source and replica server are joined to the same domain. You will also need to configure vSwitches on the replica server to match those on the source server, or ensure you reconfigure the vSwitches associated with the NICs on individual VMs.

Enabling Replication

The first step to setting up replication is enabling the destination server (replica server) to receive replicas and ensure that the proper firewall ports are opened on both the Windows firewall and any firewall appliance that sits between the source Hyper-V server and the replica server. From the Hyper-V MMC choose Hyper-V settings on the right. Once the dialog Window appears click Replication Configuration, then check boxes to enable this computer as a replica server using Kerberos and HTTP.

enablereplica

Next we will need to ensure that our inbound rules allow for inbound replication. From control panel open Windows Firewall with Advanced Security and allow the following rules (if not already enabled)

firewall

Enabling Replication on VMs

enable replication

Now that we have setup the replica server we can proceed to the source Hyper-V host and setup VM replication. To enable VM replication right click the VM and choose enabled replication. At this point you will get a enable replication wizard, proceed through the wizard entering the hostname of the replica server, choosing to HTTP port 80 with Kerberos, choose to sync all VHDs or only specific ones, set replication frequency, and choose the initial replication method. Once this occurs your initial replica will begin to sync. Once the initial replication is done the VM will then sync deltas at the replication window you specified in the wizard to enable replication. From this point you can keep tabs on the replication in the replication tab at the bottom of the hyper-v MMC window for the VM you have highlighted.

 

Planned Failover

To initiate a planned failover event (either for testing, DR needs, or migration) you will want to make sure a few things are in place. Setup the Hyper-V source server as a replica server following the steps above, this step is needed if you will need to failback to the source server. Next shutdown the VM you are failing over, then right click the VM, choose replication, and then planned failover.

planned failover

Next you will see a planned failover screen, click the failover button to begin failing over.

planned failover2

If you are performing a one way migration you can safely ignore warnings about failback, once the last replication window is complete after shutdown, you can disable replication from the replica side and start the VM.

 

Reverse Replication

To failback from the replica server to the source Hyper-V host, you can shutdown the VM, then once shutdown you can right click the VM on the replica server, choose replication then reverse replication and proceed through the prompts. This will perform a last data sync before failing back so any changes made since the failover event will be sync’d back to your production server.

4718.U3_6B177F2C

Summary

Hyper-V replication is a fantastic tool that is super simple to setup and makes setting up a warm standby painless. A few notes of caution, the first being that replication is not a suitable replacement for backups. Replication is good for DR in the sense that your RTO will be low with minimal data loss, however this doesn’t change the need to be able to retrieve old files, or restore data in the event of a catastrophic server failure or attack, as these changes are quickly replicated from the source server to the replica server. The second note of caution is that if you are using Microsoft Failover Clustering with Hyper-V you will need to setup a Hyper-V Replica Broker as a failover cluster role (only one needed per cluster) in order to act on behalf of the cluster to replicate VMs and in and out of the failover cluster. There are some caveats to this that I will document in a future blog post, however despite the extra steps needed for this it is still a fairly painless process and Hyper-V clusters can replicate to standalone Hyper-V hosts and vice versa.