Nagios

In our new VMware View and vCenter infrastructure, we're humming along, although in a matter of three weeks, we're nearly at capacity with our current hardware infrastructure (mostly in memory and disk space usage). As a result of our heavily loaded infrastructure, we've realized the criticality of setting up a monitoring system to monitor our servers and infrastructure components.

My previous experience has been limited to using Microsoft SCOM and I was not very impressed; granted I did not have the opportunity to really learn its ins and outs, I just know that we constantly had rogue alerts, false positives, and SCOM server problems (although these issues could very well be attributed to how it was set up, not the product itself).

Furthermore, our infrastructure is primarily Linux-based (RHEL Oracle server and ESX hosts), so why pay for a Microsoft tool designed to really monitor primarily Windows servers when we have the world of open-source available to us? Enter the picture Nagios, a free, open-source IT infrastructure monitor. Like many open source projects (think Red Hat, SLES, etc) the "core" is available for free and there is a pay-for commercial version. With our limited budget and no need for elaborate performance metrics and reports, we opted for Nagios Core. While it requires a decent amount of time to set up the service and host definitions, install the correct plugins, and configure the client monitors, we've been very pleased with it as a monitoring solution. It's pretty straightforward to set up and tweak, we can customize and adjust alert levels for individual hosts. And thanks to the engineers over at op5 (a monitor based heavily on Nagios core), we can monitor our VMware virtual datacenter with Nagios. While we're still waiting for our SMTP relay to be configured (the client wants ownership), so I can't opine on the email alert functionality, thus far we've been very pleased.

RHEL iSCSI part deux

So as it turns out, the earlier post was not always the best approach to using iSCSI with RHEL. An important omission from the earlier post is that we are using Dell EqualLogic storage, and thus installed the EqualLogic Host Integration Tools (HIT). These tools help integrate iSCSI and multipathing into the server, changing the approach in the process. Below are the steps that we took to get our EqualLogic iSCSI storage online and working:

1. Create a new volume using the administration console of the SAN and ensure that a CHAP user has been created that has access to the volume


2. Install equallogic-host-tools on the server.


3. As root (or using sudo), ensure that ehcmd is started service ehcmd status


4. Edit /etc/iscsi/iscsi.conf by setting node startup as manual (as otherwise, iSCSI will try to establish too many redundant connections):
   
   #***************** # Startup settings #***************** #
   To request that the iscsi initd scripts startup a session set to "automatic". # node.startup = automatic # # To manually startup the session set to "manual". The default is automatic.
   node.startup = manual
   
   

   enable CHAP and setting the CHAP user and password

   
   
   # ************* # CHAP Settings # ************* #
   To enable CHAP authentication set node.session.auth.authmethod # to CHAP. The default is None.
   node.session.auth.authmethod = CHAP
   # To set a CHAP username and password for initiator # authentication by the target(s), uncomment the following lines:
   node.session.auth.username = user
   node.session.auth.password = password
   
   

   ...

   
   

   # To set a discovery session CHAP username and password for the initiator # authentication by the target(s), uncomment the following lines: discovery.sendtargets.auth.username = user discovery.sendtargets.auth.password = password



5. Run eqltune -v and follow any recommendations made by the tool


6. run ehcmcli -d to view the ehcmcli diagnostics information


7. Check to see which subnets EqualLogic host connection manager will manage. # rswcli -L. Exclude any subnets that you do not wish to use with multipath. In our case, we have two interfaces for data on one subnet and two interfaces for iSCSI on two other interfaces: rswcli -E -network 10.52.195.128 -mask 255.255.255.128 Run rswcli -L and ehcmcli -d to confirm that the correct adapters/interfaces are being used


8. Run Discovery:
   # iscsiadm -m discovery -p 10.10.5.50 -t st 10.10.5.50
   
   
   
   
   iqn.2001-05.com.equallogic:0-8a0906-a884a4502-fdefc658d2e4d064-oracle-data 10.10.5.50:3260,1
   
   
   iqn.2001-05.com.equallogic:0-8a0906-a884a4502-fdefc658d2e4d064-oracle-data 10.10.5.50:3260,1
   
   
   iqn.2001-05.com.equallogic:0-8a0906-a884a4502-fdefc658d2e4d064-oracle-data 10.10.5.50:3260,1
   
   
   iqn.2001-05.com.equallogic:0-8a0906-a884a4502-fdefc658d2e4d064-oracle-data 10.10.5.50:3260,1
   
   Multiple identical results will be returned as a result of the intefaces that the EqualLogic daemon creates.


9. Login to the target specifying the interface:
   # iscsiadm -m node -T iqn.2001-05.com.equallogic:0-8a0906-a884a4502-fdefc658d2e4d064-oracle-data -I eql.eth1_0 -l


10. Allow automatic logins using the node.startup property. Recall that this was set to manual in the /etc/iscsi/iscsi.conf file in the steps above, since we don't want to open sessions on all interfaces because the EqualLogic daemon creates multiple interfaces using the same adapters (eql.eth1_0, eql.eth1_1, etc). Instead, we allowed automatic startup on two interfaces, one for each adapter (eql.eth1_0 and eql.eth3_0):
    # iscsiadm –m node –T -I eql.eth1_0 –o update –n node.startup –v automatic
    
    
    # iscsiadm –m node –T -I eql.eth3_0 –o update –n node.startup –v automatic


11. Edit /etc/equallogic/eql.conf [MPIO Autologins] section to specify which adapters to use:
    
    # Autologin configuration [MPIO Autologins]
    
    
    # ehcmd will update any targets configured for autologin so the boot time logins
    # are only made through two iscsiadm ifaces using independent ethernet ports.
    # The ehcmd will create additional iSCSI sessions if necessary at a later point.
    # # If you wish to control which ethernet interfaces are used for autologin,
    # list the desired interfaces below, using a separate line for each.
    # Example: # Port = eth0
    # Port = eth1
    Port = eth1
    Port = eth3




12. Create a physical volume using the /dev/mapper/ name, not the SD device name or one of the derivatives created by the equallogic daemon (eql-0-8a0906-d11b93e09-93720a256664d91d-a, eql-0-8a0906-d11b93e09-93720a256664d91d-oracle-datap1, etc) # pvcreate /dev/mapper/eql-0-8a0906-d11b93e09-93720a256664d91d-oracle-data


13. Create the volume group on the newly created physical volume. # vgcreate SANVolGroup00 /dev/mapper/eql-0-8a0906-d11b93e09-93720a256664d91d-oracle-data


14. Create the logical volume. In our case, we created the volume the entire size of the SAN volume: # lvcreate --extents 100%VG --name ora_data2 SANVolGroup00


15. Format the file system as ext3: # mkfs.ext3 /dev/SANVolGroup00/ora_data2


16. Mount the new lv to a mount point: # mount /dev/SANVolGroup00/ora_data2 /oracle/ora_data2


17. Finally, edit /etc/fstab to mount the logical volume on startup: /dev/SANVolGroup00/ora_data2 /oracle/ora_data2 ext3 _netdev,defaults 0 0 Ensure that the mount option _netdev is used, so the volume does not mount until network interfaces are up

RHEL Mapper stealing Filesystem

Today, we've setup our RHEL server to use iSCSI with CHAP authentication to connect to our iSCSI array volumes. We found this tutorial very helpful, although instead of open-iscsi, we simply used RedHat's iscsi-initiator-utils package (probably the same thing).

One thing to note is that after successfully authenticating to the array via CHAP and partitioning the block (which was /dev/sdd) with LVM2, we could not successfully run the pvcreate /dev/sdd1 command to create a physical volume on which we can build our LVMs. When running pvcreate, we'd get the following:

#pvcreate /dev/sdd1
Can't open /dev/sdd1 exclusively. Mounted filesystem?

Although running 'mount' wouldn't show the drive mapped anywhere. As it turns out, after writing LVM2 to the partition table, /dev/mapper grabs onto the device which effectively "mounts" it within the mapper. In our case, it was listed in the mapper as three devices:

eql-0-8a0906-8a0b93e09-01120a256614d8a6-ora-data01p1
eql-0-8a0906-8a0b93e09-01120a256614d8a6-ora-data01
eql-0-8a0906-8a0b93e09-01120a256614d8a6_a

Note the long name referencing our Equillogic iSCSI SAN. We found two workarounds:

1. Run pvcreate using the /dev/mapper/ name, so:
   #pvcreate /dev/mapper/eql-0-8a0906-8a0b93e09-01120a256614d8a6-ora-data01p1
   This was less desirable, however, because of the long file name
   
2. Remove the device from the /dev/mapper and then run pvcreate:
   
   #dmsetup ls
   #dmsetup remove eql-0-8a0906-8a0b93e09-01120a256614d8a6-ora-data01p1
   #dmsetup remove eql-0-8a0906-8a0b93e09-01120a256614d8a6-ora-data01
   #dmsetup remove eql-0-8a0906-8a0b93e09-01120a256614d8a6_a
   #pvcreate /dev/sdd1
   Physical volume "/dev/sdd1" successfully created
   
   

That solved our problem, allowing us to create the physical volume and then create our LVMs on top of it. Thanks to this forum article for the tip.

Active Directory Port Requirements

Here on the client site, we're in the process of getting our VMWare View 4.6 environment up and working. As one of the required steps, we need to get our vCenter and View Connection Servers on the domain. It so happens that our client runs their own domain (as much as I'd like to run our own), so we've had to coordinate with them for this effort. When trying to join our server to the domain, the servers (which are Windows 2003) are able to query DNS, resolved the FQDN, and identify the Domain Controllers, however we could never get to the point where we're prompted to join our servers. Instead, we receive the following error:

DNS was successfully queried for the service location (SRV) resource record
used to locate an Active Directory Domain Controller for domain local:

The query was for the SRV record for _ldap._tcp.dc._msdcs.local

The following AD DCs were identified by the query:

server1.fqdn
server2.fqdn

Common causes of this error include:

- Host (A) records that map the name of the AD DCs to its IP addresses are
missing or contain incorrect addresses.

- Active Directory Domain Controllers registered in DNS are not connected to
the network or are not running.

We initially thought the error to be related to WINS as TCP port 137 was closed from our VLAN to the client's VLAN that hosts the Domain Controller (yes, even in a 2008 AD Environment, the client still uses WINS). We ensured that TCP 389 was open by using telnet and confirmed that it was. As it turns out, that wasn't the issue. We found that although TCP 389 was allowed, UDP 389 was being blocked. After enabling UDP 389, we were able to query AD fine and join the domain.

This was a bit perplexing to us as I've never known AD to require UDP 389, but perhaps that is because its always been open for me. After a bit of Googling, I learned that LDAP only uses UDP 389 when querying the domain to find the list of domain controllers. After querying DNS and getting a list of possible DCs, the client's netlogon service sends a datagram to each domain controller in the form of a UDP LDAP packet, over UDP 389. All available domain controllers respond with the information for DcGetDCName. The client's netlogon service then caches this information to prevent having to constatly query the DCs. Thanks to Marc Nivens at Experts-Exchange for the tip [link].

So long story short, open TCP AND UDP 389 for LDAP/Active Directory.

Addendum

As noted above, after we had opened UDP 389, we were able to get a credential prompt, however after entering correct credentials, we would still receive some odd errors and could not join the server to the domain. After a bit of Googling, we found this Microsoft Technet article outlines the ports required for 2003 and 2008 domains, most notably the high-level ports.

Lesson Learned

After not eating lunch today and doing some quick configurations withing vCenter, I accidentally deleted a vDisk off of the wrong VM. Not good. Sure enough, when logging into the ESX host, the vDisk was gone, unrecoverable. I recall VMWare ESX 3.5 had a utility named vmfs-undelete for the purpose of recovering deleted vmdk files. No longer:

vmfs-undelete utility is not available for ESX/ESXi 4.0
ESX/ESXi 3.5 Update 3 included a utility called vmfs-undelete, which could be used to recover deleted .vmdk files. This utility is not available with ESX/ESXi 4.0.

Workaround: None. Deleted .vmdk files cannot be recovered.

[VMware ESX 4.0 Update 1 Release Notes]

Fortunately the data on the vDisk can be recreated by the developer in about an hour, so all told we're not set back too far. An important lesson learned though:

1. Always double check before permanently deleting
2. Don't have multiple configuration windows open at the same time
3. Eat lunch

The perils of using Wifi for Mobile Geographic Positioning

It is pretty well known that the Google StreetView cars that capture the images used for Google Street also collect information about Wifi networks in the area, to improve its location based services. In the past, Google has been known to have been collection too much information on private Wifi networks. That aside, it is a pretty cool concept- if your mobile device, such as an ipod touch, lacks GPS capabilities, it can still find where you are based on the Wifi networks around you.

The ingenuity of their system is also their crux. Take for example what I learned today. I recently made the jump to a smartphone, an HTC Evo. I LOVE the phone and 4G, one of the primary reasons I opted for the phone was for its location-based capabilities. This weekend, I am down at the family beach house enjoying a week away from work and getting in some riding/hiking/sleeping time. However I've noticed recently that when in the house, the phone's weather report is mapping Athens, GA as my current location, although I am 250+ miles south. I wrote this inaccuracy off as a glitch in Sprint's towers or something along those lines, that is until I took a closer look at Google Maps on my phone today.

I found it curious that the phone was putting me in Athens, which is where I attended the University of Georgia for four years. Out of curiosity I wanted to see where in Athens it was putting me, as I'm pretty familiar with the city and its streets. After zooming in a bit, I noticed that it was mapping me near the intersection of Baxter and Milledge ave. After zooming in all the way (which is not as far as I've been able to do when GPS is turned on), I realized where my phone was pinpointing me- off of Springdale Street, which is where I lived for two years while at the University. Sure enough, my phone was using Google's massive database of Wifi data (MAC, SSID, etc) to locate me, the same place where it had captured the router's MAC 3-4 years ago. Although the SSID of the router has changed (as well as the encryption key, etc), it is obvious that Google maps is using the MAC (media access control) address of the router as its primary identifier- it makes sense, as the primary purpose of MAC addresses is to be a unique identifier. So although I am not really near Athens at all, I am near the router that was in Athens at the time Google captured its MAC.

This presents somewhat of a quandry using Google maps. I guess Google just didn't count on people moving very often, or they figured they update their database frequently enough to mitigate this problem (note that it has been 3 years since the router as resided at that location, it has since changed geographies twice). It begs the question; is there somewhere I can go to update their database to reflect the router's new location? Also, would this occur no matter where in the world the router was located, so long as there were not any indexed networks nearby? Regardless, it is an illustrative example of how Google Maps works using Wifi data and how this clever system also has its limitations.

Gnome Wallpaper Changer

One feature I really liked about Windows 7 that is lacking on F13 Gnome is the ability to automatically rotate desktop wallpapers. This minor aesthetic functionality is easily implemented using crontab and a simple python script. I set the script to run every 15 minutes using crontab and to read from a specific folder of wallpapers (.backgrounds/dual). I used Davyd's Madeley's python script:

#!/usr/bin/python
#
# change-background.py
#
# A script to change to a random background image
#
# (c) 2004, Davyd Madeley
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software Foundation,
# Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#

backgrounds = ".backgrounds/dual"

import gconf
import os
import random
import mimetypes

class GConfClient:
def __init__ (self):
self.__client__ = gconf.client_get_default ()
def get_background (self):
return self.__client__.get_string ("/desktop/gnome/background/picture_filename")
def set_background (self, background):
self.__client__.set_string ("/desktop/gnome/background/picture_filename", background)

client = GConfClient ()


dir_items = os.listdir (os.path.join (os.environ["HOME"], backgrounds))
items = []

for item in dir_items:
mimetype = mimetypes.guess_type (item)[0]
if mimetype and mimetype.split ('/')[0] == "image":
items.append (item)

item = random.randint (0, len (items) - 1)
current_bg = client.get_background ()

while (items[item] == current_bg):
item = random.randint (0, len (items) - 1)

client.set_background (os.path.join (os.environ["HOME"], backgrounds, items[item]))

The crontab file is set as:

*/15 * * * * DISPLAY=:0.0 /usr/local/bin/change-background.py

Citrix VDI

Haven't had much time to post lately as we're deep in our Citrix VDI build at work. Our lead VDI engineer left the firm about the same time our hardware arrived, which means I am not the full time engineer on the project, which I'm sincerely enjoying as it affords me the opportunity to learn and really familiarize myself with our XenDesktop 4, PVS 5.6 SP1, XenApp 6 environment. The downside is I definitely am caught spinning my wheels sometimes simply because I'm not that familiar with the technology. As a result, below is a list of a few things that I've learned thus far that is not documented on Citrix's site that may be useful for others experiencing similar issues:

1. When installing UPM on Windows XP VDIs, change the default installation location to C:\Citrix\ProfMgr. Supposedly, it can handle 8.3 filenames (filenames like C:\Prog~\Citrix\Prof), however we never got it working in our environment, even though 8.3 filenames are enabled.
2. When working on the GoldenImage or vdisk in private mode, make sure you delete all domain user profiles before XenConverting or putting the vdisk in standard mode. Otherwise UPM will complain of log file issues as it can't write to the log file because the log file is read-only, imprinted in the user profile on the vdisk, which is read-only
   
3. XenDesktop 4 does not like MS SQL 2008 SP2. While SQL 2008 is not officially supported, we've had no issues using it until SP2 was applied. Once SP2 was applied, XenDesktop continued to work, but could not reliably determine the power state of the VDIs and the XenDesktop Setup Wizard would always fail on the PVS with an error of "Invalid Host ID" when trying to create the desktop gropu
   
4. Make sure to check the "Manage Machine Account Password" on the VDisk File properties! Otherwise you'll have a nightmare of AD issues and the VDIs won't register using the Desktop Agent to the DDC
5. In a somewhat unique environment, we are required to stream to two separate VLANs. To accomplish this, we teamed two of our physical NICs on our PVS servers and trunked the teamed connection to each VLAN. This alone will not accomplish streaming to both VLANs as both the TFTP service and Provisioning Server Two Stage Boot Process only listen on one NIC by default. If you try opening the tftp.cpl control panel, it only allows you to check one adapter for the service. As for the PVS two-stage boot process control panel (.cpl), we couldn't even get it to open on our 2008 R2 Server. To allow the services to listen on all adapters, go into the registry HKLM\System\Controlset001\services\\parameters, clear the value of the "Adapter" reg key. If you run a netstat -a you'll see the service is listening on 0.0.0.0 *.*
6. Due to a client security requirement, TFTP is disallowed in the environment. As a result, we use boot ISOs to boot to our VDisks. Through trial and error, we discovered that to boot to an empy vdisk for the first time, you must use PXE boot (and subsequently, TFTP). You cannot use a boot iso when first creating a vdisk.

Off the top of my head, that's all I can think of right now. As we continue through our build phase, I will continue to supplement this list.

Helpdesk Tickets

As I may have noted in earlier posts, on my current project I split time between two roles: Citrix Virtual Desktop Engineer and Desktop Engineer, the latter essentially a glorified helpdesk technician.

On the project, we do in fact have a "helpdesk," although not in the traditional sense. Our "helpdesk" functions more as a service desk, where the helpdesk representative doesn't actually perform any troubleshooting or support duties. These "helpdesk" personnel simply take phone calls and transcribe the issue the end-user is having into our archaic and cumbersome ticketing system, and then assigns the ticket to either me or another "Desktop Engineer". Two important and relevant points follow:

a.) the helpdesk representatives don't in fact really utilize their brains. They are simply human transcribing machines. The message is spoken to the helpdesk representative, then transcribed into our ticketing system.
b.) there exists software today, and has existed for nearly a decade, that does exactly this function... better.

Here are a few tickets I've received just today; I can't decide whether it is more sad or funny. Keep in mind that the individuals authoring these tickets are grown adults that assert they are college-educated.

9/17/2010 11:02AM user name when she put in her flash drive in her computer it is not recognized it

9/17/2010 10:20AM user name he need to have his lapt top configure for wirless in the office location

9/17/2010 11:52AM user name is will be in office location the week of September 27 she was told by team lead to sey up her computer so that she can use a wireless card.

Linux Kernel Contributors

Consistent with what I found in my research for my undergraduate economics thesis, "A Theoretical Analysis of the Efficiencies of Software Development Models," virtualization.info cites Citrix and VMWare as being ranked as the #12 and #20 (respectively) contributors to the newest Linux Kernel (2.6.34). Although this may seem initially counterintuitive that two proprietary software development firms are such large contributors to open source software, it is consistent with my research findings that the largest contributors to open source software communities are in fact private, for-profit industry firms.

openSUSE on KVM

Most recent guest OS on my RHEL KVM/Qemu setup:






OpenSUSE 11.2 with KDE 4.3. Initial impression is that KDE is pretty slick, but quite different from what I'm used to, back the days of KDE 2 and 3. In terms of usability, it definitely has a learning curve. I'm definitely partial to the simple elegance of Gnome, but I intend to play a bit more with OpenSUSE and KDE.

Creating ISO images from a Directory

Recently at work, we had a need to transfer Citrix XenDesktop installation files into our VSphere 4 lab environment, however our ESX servers did not support USB, as the hypervisor did not have usb-storage.ko, uhci-hcd.ko, or ehci-hcd.ko modules compiled and we didn't have time to compile and load them into the kernel.

From our VSphere console, we can only mount ISOs and we can't drag and drop files into the VM, like in VMWare workstation, so the easiest way to get our files into our VMs was to burn an ISO. Fortunately this is pretty easy to do in Linux:

mkisofs -o files.iso -J ./folder

Credit where credit is due, this blog post was extremely helpful.

Playing DVDs on RHEL5

As a consultant, one critical functionality on my laptop is the ability to watch DVDs- as trivial as it sounds, my laptop serves more than just the sole purpose of a work tool. Outside of work, it's my portal to the world, including personal e-mail, online television, and watching movies. Since I'm staying in hotels 4-5 days a week, I don't have the option to go home to my desktop every day after work. My firm understands this, and while the work laptop is primarily for work, the official policy allows us to use our laptops for "limited personal use," which I believe is the right approach- its more of a quality of life factor than anything else.

So after installing RHEL5 as my primary OS and virtualizing my work OS on KVM (whether this falls under "limited personal use" is somewhat murky :-), I needed to ensure that I could still watch movies on the laptop. Unfortunately unlike Ubuntu, Fedora, or Gentoo, RHEL does not have a robust media offering in the standard supported repositories. One reason being that RHEL is really designed for enterprise use, so generally, DVD playback and media handling is unnecessary. Another reason for RHEL's lack of official support for proprietary formats is due to patent and copyright issues- since RedHat markets and sells its OS and accompanying software, it cannot support potentially contentious packages such as libdvdcss.

Nonetheless, RHEL being Linux, I knew that I'd be able to install a DVD playing application, even if installed from source, however I wanted to avoid installing from source if possible simply because of dependency issues and having to write a seamless configuration file with all the right option flags set.

Since RHEL has its Fedora cousin and CentOS brother, I was confident that I could find the appropriate RPMs to install a media player. I also knew that I'd want to install xine, as I really like its simple interface, large support of proprietary and open formats, and I had had good experience with it in Ubuntu.

I began by installing the RHEL officially supported totem package as well as an rpm of libdvdcss. While both installed rather seamlessly, it appeared that totem either couldn't find or couldn't use the libdvdcss plugin. I then tried installing xine from custom packages, but this proved unsuccesfuly with too many unresolved dependencies. After a bit of googling, this Linux Questions forum post directed me to Dag Wieers (Dutch?) repository, which included Dag's custom yum repository with packages created for RHEL5. Once on the site, installation was relatively painless. To create the repository, only one command is necessary:

# rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

After that

# yum install xine

The xine package resolved all dependencies and installed cleanly, and sure enough, I can play commercial DVDs with no issues!

Windows 7 Activation Woes

Since getting KVM and QEMU up and running and installing Windows 7 on a virtual disk (a necessary evil if I'm going to use this laptop for work), I ran into some activation woes, with Windows 7 unable to activate the volume media

Windows could not be activated.
Key management services (KMS) host could not be located in domain name system (DNS), please have your system adminstrator verify that a KMS is published correctly in DNS.
Error: 0x80092328
Description:
DNS name does not exist

Unlike the days of my youth and with the help of an MSDN subscription, my media and key is actually legitimate. After a bit of searching, I found KB929826, which outlined how to fix the problem- a simple one line command to be run as administrator:

slmgr -ipk XXXX-XXXX-XXXX-XXXX-XXXX

and all issues were solved:

Automounting NTFS drives

As I cited in an earlier post, RHEL 5 still doesn't have native NTFS support. After installing the NTFS support in RHEL 5, I would still have to manually mount my drive every time I plugged it in, after receving the below error:



Not a mission critical problem, more just an annoyance. After a bit of Googling, I found this simple fix to automount the drive:

ln -s /sbin/mount.ntfs-3g /sbin/mount.ntfs

And voila- no errors and it automounts like it should in the first place.

Flip the switch from 'Broken' to 'Fixed'

The key to using KVM: ENABLE VIRTUALIZATION IN THE BIOS

In the words of the famous Homer Simpson- "Doh!"
After doing a bit more research and piecing together why I could boot into Windows, but then everything would come to grinding halt, it occured to me to check the BIOS to ensure that virtualization is enabled, as I was continuously getting an error when starting a VM: "Could not initialize KVM, will disable KVM support".

Seeing as my intel processor supports virtualization (egrep vmx --color=always /proc/cpuinfo), why wouldn't KVM start correctly? Furthermore, modprobe kvm-intel wouldn't load correctly.

After a couple of audible, "huh."s it occured to me to check the BIOS on the HP laptop to ensure CPU virtualization was enabled. SURPRISE! It wasn't. After enabling it in the BIOS and booting, and with a little more confidence, I ran into the same problem, "Could not initialize KVM, will disable KVM support." After a bit of Googling, I found that for whatever reason, a full shutdown and cold boot were necessary for the BIOS setting to take effect. Sure enough, after a reboot and starting my VM:


And with the -CDROM option pointing to my Windows 7 ISO, I made it past the infinite "setup is starting" page, and was able to install Windows 7:



As is usually the case, the issues experienced with KVM were due to user error rather than shoddy development.


So some tasks to complete to ensure that my work laptop will actually facilitate work:

• Ensure I can share files between my RHEL host and KVM guests
• Ensure that USB devices can be used on KVM guests
• Provide my work Windows 7 guest with enough disk space to accomodate all of my work documents

After a bit of consideration, I think my best approach will be to keep my guest OS on a different virtual disk image than my documents. That way I can keep a separate non-OS partition with all of my documents on it and boot my virtual disks with the -hdb option, thus allowing me to easily switch from Windows 7 to Windows XP guests.

Some links I found useful or potentially useful:
How to resize a libvirt image
QEMU Monitor Commands
QEMU USB Handling

RHEL 5 and KVM for Enterprise Notebook

So after finally successfully downloading and burning RHEL 5 Client x86_64 and obtaining a new hard drive for my notebook, I'm ready to install RHEL and test KVM.

The RHEL install was relatively painless, as Anaconda is an amazingly simple installation utility. Unfortunately, the first road bump I ran into was that the RHEL Client installation disk does not come with any virtualization package options (Xen, KVM, etc). No worries though, I was sure I could simply find the packages on the RedHat Network and push them down that way.

Small problem: RHEL didn't recognize my wireless card. After a quick lspci and some grepping:

lspci | grep wireless

I found my card and a nice step-by-step guide to install the necessary firmware to get it working.

After getting wireless up and running, I found that after registering my RH system, it did not appear in my RedHat systems on RHN, therefore I couldn't target packages to be automatically pushed to my laptop. After a bit of research on the RedHat Knowledgebase, I determined that this was because of the way my enterprise evaluation account was set up- my systems would be assigned to the Organization Administrator. Being the impatient person I am, I decided to just pull down the necessary packages manually instead of waiting for a response from my company's RHEL partner contact.

It was a bit of a tedious process, as I had to not only pull down the necessary RPMs, but also their dependencies. Anyone who's had experience working with RedHat, SuSE, or any other RPM-based distro before yum became available knows how frustrating it can be. I actually documented each package I had to pull down and listed them in the original version of this post, however Blogger didn't do a good job of saving the post, so that list has been lost in the series of tubes.

After pulling down all the packages I needed (apparently RHEL only officially supports up to version 83 of KVM) and checking the Guest Support Status, it looked as though I'd be good to go with virtualizing Windows Vista or 7 on KVM.

The first thing I tried was to convert the VMWare VMDK virtual image of my Windows Vista laptop into qemu's qcow2 format, as outlined in a link from my previous post. I quickly discovered, however, that RHEL does not natively support reading/writing to NTFS filesystems. What!? What is this, 1998? After a bit of Googling, I found this NTFS support in RHEL 5 blog post outlining the basic packages needed. One thing to note is that the dev package (kmdl) is not necessary and will generate a compile-time error if your kernel is not 2.6.18. I went ahead and just skipped installing it and was able to mount ntfs file systems fine. I figured I'd try configuring the automounter later, after I got my converted Vista image converted:

qemu-img convert Vista.vmdk -O qcow2 /home//virtual_machines/Vista.qcow2

The conversion appeared to complete successfully, however it did take a good bit of time (roughly 30 minutes).

kvm -hda Ubuntu-copy.qcow2 -net nic -net user -m 512

Upon attempting to boot this image, I received a blank screen with no indication of the guestOS trying to boot. After a couple more attempts, I did get the "Windows failed to start properly" menu and tried booting into safe mode, however this too failed. No sweat, there's a chance that my virtual image didn't convert well in the P2V, and I haven't yet tested it on VMWare, so I figured why not start from scratch with a brand new Windows 7 image?

qemu-kvm -hda win7.img -cdrom win27.iso -m 1024 -boot d

Success! (at least initially...) The Win7 ISO booted up beautifully and actually relatively quickly. I was able to run through the initial steps to install Windows 7- selecting time zone settings, user settings, etc. That was until I got to the "Setup is Starting" screen and where I was left in indefinite purgatory. I did a bit of googling and found this bugzilla report outlining the same issue I was experiencing, but with no resolution. Tracing duplicate bugs led to no where, and thus again I was stymied.

Alright, well I don't really need Windows 7, why not just use XP Professional? Being an older OS, perhaps it would install more cleanly on KVM. As before, the ISO booted great, I ran through all the OS options, and it actually started to install... only to later hang just as its younger brother did before. After a few repeated attempts (one allowed to run overnight), it was obvious that there was little I could do to get a Windows OS up and running on this version of KVM.

Alright, well maybe the version packages I chose to install KVM were incomplete or still unstable. Afterall, the kvm-qemu package was listed as in Beta. So I decided to scrap my RHEL client idea and go for the real deal: RHEL 5 Server, whose installation media actually contains virtualization packages. Despite this more by-the-book (and probably more officially supported) attempt, I ran into identical problems as noted above

So the conclusion: KVM has potential as an alternative to a baremetal hypervisor, perhaps just not version 83. RedHat is striving to make RHEV (RedHat's implementation of KVM) a market contender for a low-cost virtualization solution, however it seems this may be a bit premature. Maybe they should give their internal developers a bit more time to design and test before selling that horse.

While the battle may be lost, the war is far from over- next attempt: the more tried and true Xen!

Packages: kvm-qemu-img

Links:
HowTos/KVM - CentOS
RHEL Virtualization Guide
(dev package not required- complains if kernel is different)

P2V of Vista machine failed

P2V for KVM

Some resources I plan on using when trying to P2V a Windows Vista laptop to use on either RHEL or CentOS KVM.

• How to convert vmware vdk image to kvm
• Windows XP p2v conversion with KVM

Right now, the best solution looks to be to convert the physical box to a VM using VMWare vCenter Converter

I haven't ruled out using Xen, but my initial research indicates this may be easier to do on KVM then Xen. KVM appears to be more similar to Solaris Containers, whereas Xen uses paravirtualization. I've had limited experience with Xen, and no experience with KVM, so this should be a good learning exercise. Here's some more resources for reference:

• Xen vs KVM
• Xen vs KVM and KVM Migration
  

Managing Windows Bitlocker Drive Encryption (BDE)

A quick and easy way to manage Windows Bitlocker Drive Encryption (BDE).

C:\Users\user>manage-bde.wsf -help

Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

manage-bde[.wsf] -parameter [arguments]

Description:
Configures BitLocker Drive Encryption on disk volumes.

Parameter List:
-status Provides information about BitLocker-capable volumes.
-on Encrypts the volume and turns BitLocker protection on.
-off Decrypts the volume and turns BitLocker protection off.
-pause Pauses encryption or decryption.
-resume Resumes encryption or decryption.
-lock Prevents access to BitLocker-encrypted data.
-unlock Allows access to BitLocker-encrypted data.
-autounlock Manages automatic unlocking of data volumes.
-protectors Manages protection methods for the encryption key.
-tpm Configures the computer's Trusted Platform Module (TPM).
-ForceRecovery or -fr
Forces a BitLocker-protected OS to recover on restarts.
-ComputerName or -cn
Runs on another computer. Examples: "ComputerX", "127.0.0.1"
-? or /? Displays brief help. Example: "-ParameterSet -?"
-Help or -h Displays complete help. Example: "-ParameterSet -h"

Examples:
manage-bde -status
manage-bde -on C: -RecoveryPassword -RecoveryKey F:\
manage-bde -unlock E: -RecoveryKey F:\84E151C1...7A62067A512.bek

Making it much easier to image and P2V a machine. This will come in handy when I P2V Windows to run on virtualized hardware, so I can finally make the switch to all open source OS (read: Linux)!

Deploying an EXE patch using GPO

Recently at my client site, we needed to push a Microsoft Patch that we couldn't push using our update solution. The ideal solution would be to wrap the executable up into an MSI and just push the MSI using a GPO. After trying this using a 3rd party MSI wrapper utility, we created an MSI and pushed it to all workstations, where the wrapper itself installed, but the executable failed to run. In the interest of time, we needed to engineer a solution quickly to push our patch to ensure that users would be able to continue to access certain resources within our infrastructure. I know that you can create a Setup package by using Visual Studio .NET (another tutorial here), but we didn't have the time to sit down and hammer out a solution and test it.

Instead, I suggested that we push the executable using a GPO and simple VB script. The trick is we'd have to build into the script a way to check to make sure the executable hasn't already been installed- otherwise every computer getting the GPO will run the executable every time it boots (something we don't want). After a bit of Googling, I wrote the following VB script, which does the following:

# Pseudo Code
Check if a reg key exists
If the reg key does not exist, execute the executable installer with switches to make it a passive installer
Once the executable complete, create a registry key marking that the install has completed

'Reg key to create. Doesn't have to be this key, but since this is technically a patch, put it in Windows Update
sRegKey="HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate"
sExePath = "\\path\to\executable"
sSwitches = "/passive /warnrestart:45" 'Check available flags on Executable for options here

' Suppress error in case values do not exist
On Error Resume Next

' Check for the Reg Key Marker
sRegMarkerValue = "" ' initial value
sRegMarkerValue = oShell.RegRead( sRegKey & "\WindowsXP-KBXXXXX-x86-ENU.exe")
On Error Goto 0

' To ensure update is only installed once, test the reg key marker
If sRegMarkerValue <> "yes" then

'Run the executable with switches
oShell.Run Chr(34) & sExePath & Chr(34) & " " & sSwitches, 1, True

' Create the Reg Key marker
oShell.RegWrite sRegKey & "\WindowsXP-KBXXXXX-x86-ENU.exe", "yes"
End If

After applying the GPO, we rebooted all of the workstations, and our patch was successfully applied!