Citrix XenServer VM’s Unmanageable

Problem:

 

Xen begins throwing errors about not being able to attach VDIs, XenMotion stops working, and the system becomes unmanageable, or xapi service immediately dies after starting

 

 

Solution:

check space on dom0

df -h

If this is at a high threshold you may want to do the following:

Remove patches from /var/patch

cd /var/patch
rm <uuid of patch>

Remove old log files

rm -rf /var/log/*.gz

Clean temp log files

rm -rf /tmp/*.log

Cleanup any old patches

xe patch-clean

Clean out /var/log/btmp

echo > /var/log/btmp

Restart Xen Toolstack

xe-toolstack-restart

 

Re-occurance Prevention:

Create a mounted volume for /var/log or do the following:

in /etc/cron.daily create the following script called logclean.sh

#!/bin/bash
rm -rf /var/log/*.gz

Then change permissions to make it executable

chmod 700/etc/cron.daily/logclean.sh

Another alternative would be to setup an NFS mount in /var/log, however I already use a centralized log aggregation solution so this cron job while a fairly blunt instrument meets my needs.

Unable to Manage VMs in Hyper-V

Recently I came across a situation where a set of VMs running on cluster shared storage were all in an un-manageable state. They were not listing as being present on the host and exhibited the error shown in symptoms:

Symptoms

Virtual Machines show in a running or paused state but have the following error when attempting to access console, settings, or take any other management actions:

Screen Shot 2015-12-26 at 12.45.36 PM

Additionally try listing the VMs in Powershell

Get-VM

If the VM does not show in the list it is definitely in this state.

Problem

Check to see if any other VMs exhibit this issue and if they are on cluster shared storage. If you are using it make an exclusion for the Cluster Volume or disable AV as this is usually the culprit.

Remediation

Install Sysinternals process explorer here: https://technet.microsoft.com/en-us/sysinternals/processexplorer.aspx

Next we will locate the GUID of the VM in question

Screen Shot 2015-12-26 at 12.10.36 AM

Once you have located the GUID open process explorer as administrator and locate right click the VWP.exe processes and check the commandline entry under the “image” tab. This will display the GUID of the VMs. Select the VWP.exe that matches your trouble VM and kill the process.

DO NOT kill any other processes or you will potentially bring down other VMs

Screen Shot 2015-12-26 at 12.10.23 AM

Once you have done this you will need to restart the Hyper-V management stack by restarting the VMMS process. I have found some of these will not work properly with the restart-service cmdlet, so you will need to stop and then start the service. This does not effect the running VMs, it only effects your ability to manage them while this service is stopped.

Get-Service vmms
Stop-Service vmms
Start-Service vmms
Get-Service vmms

Now you should be up and running!

Enabling PXE Boot for Citrix XenServer VMs

Seasons greetings! i’ve been doing some testing with Cobbler deployment in my lab and ran into a minor snag getting Citrix XenServer VMs to PXE boot. Below are the steps to enable boot options for XenServer Guest VMs to utilize PXE:

 

List VMs

List VMs and make note of the UUID for the VM in question

xe vm-list

 

Setting HVM Boot Options

Using the UUID from the command above change the boot options to enable HVM mode booting and set the boot order to disk then network

xe vm-param-set uuid=<your UUID> HVM-boot-policy="BIOS order"
xe vm-param-set uuid=<your UUID HVM-boot-params:order=cn

 

Start The VM

xe vm-start uuid=<your UUID