Citrix VDI

Haven't had much time to post lately as we're deep in our Citrix VDI build at work. Our lead VDI engineer left the firm about the same time our hardware arrived, which means I am not the full time engineer on the project, which I'm sincerely enjoying as it affords me the opportunity to learn and really familiarize myself with our XenDesktop 4, PVS 5.6 SP1, XenApp 6 environment. The downside is I definitely am caught spinning my wheels sometimes simply because I'm not that familiar with the technology. As a result, below is a list of a few things that I've learned thus far that is not documented on Citrix's site that may be useful for others experiencing similar issues:

1. When installing UPM on Windows XP VDIs, change the default installation location to C:\Citrix\ProfMgr. Supposedly, it can handle 8.3 filenames (filenames like C:\Prog~\Citrix\Prof), however we never got it working in our environment, even though 8.3 filenames are enabled.
2. When working on the GoldenImage or vdisk in private mode, make sure you delete all domain user profiles before XenConverting or putting the vdisk in standard mode. Otherwise UPM will complain of log file issues as it can't write to the log file because the log file is read-only, imprinted in the user profile on the vdisk, which is read-only
   
3. XenDesktop 4 does not like MS SQL 2008 SP2. While SQL 2008 is not officially supported, we've had no issues using it until SP2 was applied. Once SP2 was applied, XenDesktop continued to work, but could not reliably determine the power state of the VDIs and the XenDesktop Setup Wizard would always fail on the PVS with an error of "Invalid Host ID" when trying to create the desktop gropu
   
4. Make sure to check the "Manage Machine Account Password" on the VDisk File properties! Otherwise you'll have a nightmare of AD issues and the VDIs won't register using the Desktop Agent to the DDC
5. In a somewhat unique environment, we are required to stream to two separate VLANs. To accomplish this, we teamed two of our physical NICs on our PVS servers and trunked the teamed connection to each VLAN. This alone will not accomplish streaming to both VLANs as both the TFTP service and Provisioning Server Two Stage Boot Process only listen on one NIC by default. If you try opening the tftp.cpl control panel, it only allows you to check one adapter for the service. As for the PVS two-stage boot process control panel (.cpl), we couldn't even get it to open on our 2008 R2 Server. To allow the services to listen on all adapters, go into the registry HKLM\System\Controlset001\services\\parameters, clear the value of the "Adapter" reg key. If you run a netstat -a you'll see the service is listening on 0.0.0.0 *.*
6. Due to a client security requirement, TFTP is disallowed in the environment. As a result, we use boot ISOs to boot to our VDisks. Through trial and error, we discovered that to boot to an empy vdisk for the first time, you must use PXE boot (and subsequently, TFTP). You cannot use a boot iso when first creating a vdisk.

Off the top of my head, that's all I can think of right now. As we continue through our build phase, I will continue to supplement this list.

Helpdesk Tickets

As I may have noted in earlier posts, on my current project I split time between two roles: Citrix Virtual Desktop Engineer and Desktop Engineer, the latter essentially a glorified helpdesk technician.

On the project, we do in fact have a "helpdesk," although not in the traditional sense. Our "helpdesk" functions more as a service desk, where the helpdesk representative doesn't actually perform any troubleshooting or support duties. These "helpdesk" personnel simply take phone calls and transcribe the issue the end-user is having into our archaic and cumbersome ticketing system, and then assigns the ticket to either me or another "Desktop Engineer". Two important and relevant points follow:

a.) the helpdesk representatives don't in fact really utilize their brains. They are simply human transcribing machines. The message is spoken to the helpdesk representative, then transcribed into our ticketing system.
b.) there exists software today, and has existed for nearly a decade, that does exactly this function... better.

Here are a few tickets I've received just today; I can't decide whether it is more sad or funny. Keep in mind that the individuals authoring these tickets are grown adults that assert they are college-educated.

9/17/2010 11:02AM user name when she put in her flash drive in her computer it is not recognized it

9/17/2010 10:20AM user name he need to have his lapt top configure for wirless in the office location

9/17/2010 11:52AM user name is will be in office location the week of September 27 she was told by team lead to sey up her computer so that she can use a wireless card.

Linux Kernel Contributors

Consistent with what I found in my research for my undergraduate economics thesis, "A Theoretical Analysis of the Efficiencies of Software Development Models," virtualization.info cites Citrix and VMWare as being ranked as the #12 and #20 (respectively) contributors to the newest Linux Kernel (2.6.34). Although this may seem initially counterintuitive that two proprietary software development firms are such large contributors to open source software, it is consistent with my research findings that the largest contributors to open source software communities are in fact private, for-profit industry firms.

openSUSE on KVM

Most recent guest OS on my RHEL KVM/Qemu setup:






OpenSUSE 11.2 with KDE 4.3. Initial impression is that KDE is pretty slick, but quite different from what I'm used to, back the days of KDE 2 and 3. In terms of usability, it definitely has a learning curve. I'm definitely partial to the simple elegance of Gnome, but I intend to play a bit more with OpenSUSE and KDE.

Creating ISO images from a Directory

Recently at work, we had a need to transfer Citrix XenDesktop installation files into our VSphere 4 lab environment, however our ESX servers did not support USB, as the hypervisor did not have usb-storage.ko, uhci-hcd.ko, or ehci-hcd.ko modules compiled and we didn't have time to compile and load them into the kernel.

From our VSphere console, we can only mount ISOs and we can't drag and drop files into the VM, like in VMWare workstation, so the easiest way to get our files into our VMs was to burn an ISO. Fortunately this is pretty easy to do in Linux:

mkisofs -o files.iso -J ./folder

Credit where credit is due, this blog post was extremely helpful.

Playing DVDs on RHEL5

As a consultant, one critical functionality on my laptop is the ability to watch DVDs- as trivial as it sounds, my laptop serves more than just the sole purpose of a work tool. Outside of work, it's my portal to the world, including personal e-mail, online television, and watching movies. Since I'm staying in hotels 4-5 days a week, I don't have the option to go home to my desktop every day after work. My firm understands this, and while the work laptop is primarily for work, the official policy allows us to use our laptops for "limited personal use," which I believe is the right approach- its more of a quality of life factor than anything else.

So after installing RHEL5 as my primary OS and virtualizing my work OS on KVM (whether this falls under "limited personal use" is somewhat murky :-), I needed to ensure that I could still watch movies on the laptop. Unfortunately unlike Ubuntu, Fedora, or Gentoo, RHEL does not have a robust media offering in the standard supported repositories. One reason being that RHEL is really designed for enterprise use, so generally, DVD playback and media handling is unnecessary. Another reason for RHEL's lack of official support for proprietary formats is due to patent and copyright issues- since RedHat markets and sells its OS and accompanying software, it cannot support potentially contentious packages such as libdvdcss.

Nonetheless, RHEL being Linux, I knew that I'd be able to install a DVD playing application, even if installed from source, however I wanted to avoid installing from source if possible simply because of dependency issues and having to write a seamless configuration file with all the right option flags set.

Since RHEL has its Fedora cousin and CentOS brother, I was confident that I could find the appropriate RPMs to install a media player. I also knew that I'd want to install xine, as I really like its simple interface, large support of proprietary and open formats, and I had had good experience with it in Ubuntu.

I began by installing the RHEL officially supported totem package as well as an rpm of libdvdcss. While both installed rather seamlessly, it appeared that totem either couldn't find or couldn't use the libdvdcss plugin. I then tried installing xine from custom packages, but this proved unsuccesfuly with too many unresolved dependencies. After a bit of googling, this Linux Questions forum post directed me to Dag Wieers (Dutch?) repository, which included Dag's custom yum repository with packages created for RHEL5. Once on the site, installation was relatively painless. To create the repository, only one command is necessary:

# rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

After that

# yum install xine

The xine package resolved all dependencies and installed cleanly, and sure enough, I can play commercial DVDs with no issues!

Windows 7 Activation Woes

Since getting KVM and QEMU up and running and installing Windows 7 on a virtual disk (a necessary evil if I'm going to use this laptop for work), I ran into some activation woes, with Windows 7 unable to activate the volume media

Windows could not be activated.
Key management services (KMS) host could not be located in domain name system (DNS), please have your system adminstrator verify that a KMS is published correctly in DNS.
Error: 0x80092328
Description:
DNS name does not exist

Unlike the days of my youth and with the help of an MSDN subscription, my media and key is actually legitimate. After a bit of searching, I found KB929826, which outlined how to fix the problem- a simple one line command to be run as administrator:

slmgr -ipk XXXX-XXXX-XXXX-XXXX-XXXX

and all issues were solved:

Automounting NTFS drives

As I cited in an earlier post, RHEL 5 still doesn't have native NTFS support. After installing the NTFS support in RHEL 5, I would still have to manually mount my drive every time I plugged it in, after receving the below error:



Not a mission critical problem, more just an annoyance. After a bit of Googling, I found this simple fix to automount the drive:

ln -s /sbin/mount.ntfs-3g /sbin/mount.ntfs

And voila- no errors and it automounts like it should in the first place.

Flip the switch from 'Broken' to 'Fixed'

The key to using KVM: ENABLE VIRTUALIZATION IN THE BIOS

In the words of the famous Homer Simpson- "Doh!"
After doing a bit more research and piecing together why I could boot into Windows, but then everything would come to grinding halt, it occured to me to check the BIOS to ensure that virtualization is enabled, as I was continuously getting an error when starting a VM: "Could not initialize KVM, will disable KVM support".

Seeing as my intel processor supports virtualization (egrep vmx --color=always /proc/cpuinfo), why wouldn't KVM start correctly? Furthermore, modprobe kvm-intel wouldn't load correctly.

After a couple of audible, "huh."s it occured to me to check the BIOS on the HP laptop to ensure CPU virtualization was enabled. SURPRISE! It wasn't. After enabling it in the BIOS and booting, and with a little more confidence, I ran into the same problem, "Could not initialize KVM, will disable KVM support." After a bit of Googling, I found that for whatever reason, a full shutdown and cold boot were necessary for the BIOS setting to take effect. Sure enough, after a reboot and starting my VM:


And with the -CDROM option pointing to my Windows 7 ISO, I made it past the infinite "setup is starting" page, and was able to install Windows 7:



As is usually the case, the issues experienced with KVM were due to user error rather than shoddy development.


So some tasks to complete to ensure that my work laptop will actually facilitate work:

• Ensure I can share files between my RHEL host and KVM guests
• Ensure that USB devices can be used on KVM guests
• Provide my work Windows 7 guest with enough disk space to accomodate all of my work documents

After a bit of consideration, I think my best approach will be to keep my guest OS on a different virtual disk image than my documents. That way I can keep a separate non-OS partition with all of my documents on it and boot my virtual disks with the -hdb option, thus allowing me to easily switch from Windows 7 to Windows XP guests.

Some links I found useful or potentially useful:
How to resize a libvirt image
QEMU Monitor Commands
QEMU USB Handling

RHEL 5 and KVM for Enterprise Notebook

So after finally successfully downloading and burning RHEL 5 Client x86_64 and obtaining a new hard drive for my notebook, I'm ready to install RHEL and test KVM.

The RHEL install was relatively painless, as Anaconda is an amazingly simple installation utility. Unfortunately, the first road bump I ran into was that the RHEL Client installation disk does not come with any virtualization package options (Xen, KVM, etc). No worries though, I was sure I could simply find the packages on the RedHat Network and push them down that way.

Small problem: RHEL didn't recognize my wireless card. After a quick lspci and some grepping:

lspci | grep wireless

I found my card and a nice step-by-step guide to install the necessary firmware to get it working.

After getting wireless up and running, I found that after registering my RH system, it did not appear in my RedHat systems on RHN, therefore I couldn't target packages to be automatically pushed to my laptop. After a bit of research on the RedHat Knowledgebase, I determined that this was because of the way my enterprise evaluation account was set up- my systems would be assigned to the Organization Administrator. Being the impatient person I am, I decided to just pull down the necessary packages manually instead of waiting for a response from my company's RHEL partner contact.

It was a bit of a tedious process, as I had to not only pull down the necessary RPMs, but also their dependencies. Anyone who's had experience working with RedHat, SuSE, or any other RPM-based distro before yum became available knows how frustrating it can be. I actually documented each package I had to pull down and listed them in the original version of this post, however Blogger didn't do a good job of saving the post, so that list has been lost in the series of tubes.

After pulling down all the packages I needed (apparently RHEL only officially supports up to version 83 of KVM) and checking the Guest Support Status, it looked as though I'd be good to go with virtualizing Windows Vista or 7 on KVM.

The first thing I tried was to convert the VMWare VMDK virtual image of my Windows Vista laptop into qemu's qcow2 format, as outlined in a link from my previous post. I quickly discovered, however, that RHEL does not natively support reading/writing to NTFS filesystems. What!? What is this, 1998? After a bit of Googling, I found this NTFS support in RHEL 5 blog post outlining the basic packages needed. One thing to note is that the dev package (kmdl) is not necessary and will generate a compile-time error if your kernel is not 2.6.18. I went ahead and just skipped installing it and was able to mount ntfs file systems fine. I figured I'd try configuring the automounter later, after I got my converted Vista image converted:

qemu-img convert Vista.vmdk -O qcow2 /home//virtual_machines/Vista.qcow2

The conversion appeared to complete successfully, however it did take a good bit of time (roughly 30 minutes).

kvm -hda Ubuntu-copy.qcow2 -net nic -net user -m 512

Upon attempting to boot this image, I received a blank screen with no indication of the guestOS trying to boot. After a couple more attempts, I did get the "Windows failed to start properly" menu and tried booting into safe mode, however this too failed. No sweat, there's a chance that my virtual image didn't convert well in the P2V, and I haven't yet tested it on VMWare, so I figured why not start from scratch with a brand new Windows 7 image?

qemu-kvm -hda win7.img -cdrom win27.iso -m 1024 -boot d

Success! (at least initially...) The Win7 ISO booted up beautifully and actually relatively quickly. I was able to run through the initial steps to install Windows 7- selecting time zone settings, user settings, etc. That was until I got to the "Setup is Starting" screen and where I was left in indefinite purgatory. I did a bit of googling and found this bugzilla report outlining the same issue I was experiencing, but with no resolution. Tracing duplicate bugs led to no where, and thus again I was stymied.

Alright, well I don't really need Windows 7, why not just use XP Professional? Being an older OS, perhaps it would install more cleanly on KVM. As before, the ISO booted great, I ran through all the OS options, and it actually started to install... only to later hang just as its younger brother did before. After a few repeated attempts (one allowed to run overnight), it was obvious that there was little I could do to get a Windows OS up and running on this version of KVM.

Alright, well maybe the version packages I chose to install KVM were incomplete or still unstable. Afterall, the kvm-qemu package was listed as in Beta. So I decided to scrap my RHEL client idea and go for the real deal: RHEL 5 Server, whose installation media actually contains virtualization packages. Despite this more by-the-book (and probably more officially supported) attempt, I ran into identical problems as noted above

So the conclusion: KVM has potential as an alternative to a baremetal hypervisor, perhaps just not version 83. RedHat is striving to make RHEV (RedHat's implementation of KVM) a market contender for a low-cost virtualization solution, however it seems this may be a bit premature. Maybe they should give their internal developers a bit more time to design and test before selling that horse.

While the battle may be lost, the war is far from over- next attempt: the more tried and true Xen!

Packages: kvm-qemu-img

Links:
HowTos/KVM - CentOS
RHEL Virtualization Guide
(dev package not required- complains if kernel is different)

P2V of Vista machine failed

P2V for KVM

Some resources I plan on using when trying to P2V a Windows Vista laptop to use on either RHEL or CentOS KVM.

• How to convert vmware vdk image to kvm
• Windows XP p2v conversion with KVM

Right now, the best solution looks to be to convert the physical box to a VM using VMWare vCenter Converter

I haven't ruled out using Xen, but my initial research indicates this may be easier to do on KVM then Xen. KVM appears to be more similar to Solaris Containers, whereas Xen uses paravirtualization. I've had limited experience with Xen, and no experience with KVM, so this should be a good learning exercise. Here's some more resources for reference:

• Xen vs KVM
• Xen vs KVM and KVM Migration
  

Managing Windows Bitlocker Drive Encryption (BDE)

A quick and easy way to manage Windows Bitlocker Drive Encryption (BDE).

C:\Users\user>manage-bde.wsf -help

Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

manage-bde[.wsf] -parameter [arguments]

Description:
Configures BitLocker Drive Encryption on disk volumes.

Parameter List:
-status Provides information about BitLocker-capable volumes.
-on Encrypts the volume and turns BitLocker protection on.
-off Decrypts the volume and turns BitLocker protection off.
-pause Pauses encryption or decryption.
-resume Resumes encryption or decryption.
-lock Prevents access to BitLocker-encrypted data.
-unlock Allows access to BitLocker-encrypted data.
-autounlock Manages automatic unlocking of data volumes.
-protectors Manages protection methods for the encryption key.
-tpm Configures the computer's Trusted Platform Module (TPM).
-ForceRecovery or -fr
Forces a BitLocker-protected OS to recover on restarts.
-ComputerName or -cn
Runs on another computer. Examples: "ComputerX", "127.0.0.1"
-? or /? Displays brief help. Example: "-ParameterSet -?"
-Help or -h Displays complete help. Example: "-ParameterSet -h"

Examples:
manage-bde -status
manage-bde -on C: -RecoveryPassword -RecoveryKey F:\
manage-bde -unlock E: -RecoveryKey F:\84E151C1...7A62067A512.bek

Making it much easier to image and P2V a machine. This will come in handy when I P2V Windows to run on virtualized hardware, so I can finally make the switch to all open source OS (read: Linux)!

Deploying an EXE patch using GPO

Recently at my client site, we needed to push a Microsoft Patch that we couldn't push using our update solution. The ideal solution would be to wrap the executable up into an MSI and just push the MSI using a GPO. After trying this using a 3rd party MSI wrapper utility, we created an MSI and pushed it to all workstations, where the wrapper itself installed, but the executable failed to run. In the interest of time, we needed to engineer a solution quickly to push our patch to ensure that users would be able to continue to access certain resources within our infrastructure. I know that you can create a Setup package by using Visual Studio .NET (another tutorial here), but we didn't have the time to sit down and hammer out a solution and test it.

Instead, I suggested that we push the executable using a GPO and simple VB script. The trick is we'd have to build into the script a way to check to make sure the executable hasn't already been installed- otherwise every computer getting the GPO will run the executable every time it boots (something we don't want). After a bit of Googling, I wrote the following VB script, which does the following:

# Pseudo Code
Check if a reg key exists
If the reg key does not exist, execute the executable installer with switches to make it a passive installer
Once the executable complete, create a registry key marking that the install has completed

'Reg key to create. Doesn't have to be this key, but since this is technically a patch, put it in Windows Update
sRegKey="HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate"
sExePath = "\\path\to\executable"
sSwitches = "/passive /warnrestart:45" 'Check available flags on Executable for options here

' Suppress error in case values do not exist
On Error Resume Next

' Check for the Reg Key Marker
sRegMarkerValue = "" ' initial value
sRegMarkerValue = oShell.RegRead( sRegKey & "\WindowsXP-KBXXXXX-x86-ENU.exe")
On Error Goto 0

' To ensure update is only installed once, test the reg key marker
If sRegMarkerValue <> "yes" then

'Run the executable with switches
oShell.Run Chr(34) & sExePath & Chr(34) & " " & sSwitches, 1, True

' Create the Reg Key marker
oShell.RegWrite sRegKey & "\WindowsXP-KBXXXXX-x86-ENU.exe", "yes"
End If

After applying the GPO, we rebooted all of the workstations, and our patch was successfully applied!

Windows Batch Scripting and Win2k8 AD GPO Functionality

Created this script and saved it as a .bat file to loop through values stored in a comma-delimited file to reboot specific machines:

FOR /F %%A IN (filename.csv) DO (
ECHO. %%A >> result.txt %1
shutdown -m \\%%A -r -f -t 10 -c "Your workstation will reboot in 10 seconds for scheduled maintenance" >>result.txt %1
)

And some cool functionality that you can do using GPOs in AD 2008:

• Set/Change Local User Passwords on all workstations in the domain
• Map network drives without using vb scripts/batch files
• Map printers without using vb scripts/batch files
• Disable USB storage devices using REG keys
• Disable specific hardware devices (CDROM drives, imaging devices, etc)
  

RedHat Certified Technician

I am officially a RedHat Certified Technician! Although I'm not gaining any experience in RedHat on my current project, I managed to get RedHat training approved by my firm, and took the RH133 course last week. The course was great- I learned a good deal of administration tips and tricks and it really helped fill in some crucial voids in my Linux repertoire.

I'm hoping to continue on the Red Hat path to obtain my RHCE in the next year, hopefully in October if possible. Admittedly, Gentoo still holds a special place in my heart, but in terms of usability and enterprise server-grade Linux, Red Hat has gained a lot of points in my book, particularly with its implementation yum and system configuration tools.

The class also inspired me to take a closer look at Xen and KVM virtualization. I'm hoping to P2V my Windows 7 box at home, and instead use RHEL (or CentOS) as a hypervisor and run Windows 7 as a VM.

RH courses I'm hoping to attend (and gain certification in) in the coming months include:

• RH253 - Red Hat Linux Network and Security Administration
• RH318 - Red Hat Virtualization Administration

Fast Forward 8 Months

So again, I have criminally lapsed in my posts on this blog. Since that September post, I left my previous job at the University of Georgia, had a nice 2-week backpacking trip through Europe away from computers and familiar faces, and returned in mid October to begin work for the man a large technology consulting firm, based out of the Atlanta office.

I'm currently on a 6-month 10-month project for a federal government client, spending 4-5 days of the week in the greater DC area, working as, to my dismay, a "Windows Desktop Engineer." I've been able to grab a few server tasks on the side, and the role has afforded me some valuable experience with Windows Server 2008 and Active Directory, however I find myself a good distance away from where my true interests lie: Linux servers and virtualization technologies.

At work, I've learned a good bit about GPO authoring, Active Directory 2008 configuration and management, Ghost Suite 2.5, Windows Server 2008 implementation of DHCP and DNS, enterprise-level IT, and the nature of consulting.

In what little time I have at home every few weekends (I tend to spend weekends in different cities or staying in DC), I've been playing around with Windows Server 2008 and Active Directory architecture in a virtual environment. Since leaving UGA, I had to forfeit my VMWare Workstation license, and thus have been learning to use Sun's VirtualBox. Aside from the networking, I've found that many of the standard features are comparable to VMWare Workstation.

In the coming weeks, I hope to have the opportunity to play around with Xen virtualization and KVM in Linux. More on that later!