October 13, 2018

Converting Citrix PVS Image from XenServer to vSphere

Having repeated this nightmarish migration several times now, here's the steps I've found to be most efficient:
  1. Import your XenServer-optimized PVS image (as a VHD) in to XenCenter as a new VM.
  2. Snapshot and boot the VM (just in case you mess up the next step you won't need to import again).
  3. Uninstall the Citrix PVS and Citrix Guest Tools / Xen Tools bits. 
  4. Delete xen*.sys from c:\windows\system32 and c:\windows\system32\drivers
  5. Reboot and make sure everything still comes back up. It should revert to a generic Realtek network driver.
  6. Run VMware Converter on the VM. Alternatively you can export the VM from XenCenter as an OVA and then import it to vSphere.
  7. Be sure you are using a VMXNet3 NIC on the vSphere VM, not an E1000.
  8. Boot the resulting vSphere VM and install VMware tools.
  9. Delete the ghost NIC that is left from the Realtek drivers (https://support.citrix.com/article/CTX221733), otherwise you will get the BNIstack error.
  10. Install PVS target device software and run the imaging wizard again. 
  11. Follow all your normal steps for capturing a new image
If you run in to an IRQL_NOT_EQUAL_OR_LESS BSOD, you may be like me and have some piece of software set to redirect writes to the vDisk cache disk which no longer exists. Make sure you fix that prior to attempting a migration.

The most commonly recommended solutions for BNIstack errors during your first boot after capturing the image:
  1. Make sure no ghost NICs are still present
  2. Try uninstalling any antivirus and disabling IPv6
  3. Install the hotfix for KB 2550978 (https://support.microsoft.com/en-us/help/2550978/0x0000007b-stop-error-after-you-replace-an-identical-iscsi-network-ada)
  4. Consider changing the default open retry limits/interval for BNIstack (https://discussions.citrix.com/topic/377414-bsod-with-bnistack-and-cvhdmpsys/)

Happy migrations!


December 01, 2017

DHCP Fails after uninstalling Citrix PVS Agent

While attempting to migrate a Citrix PVS base image to a new hypervisor, I uninstalled the PVS bits from the VDA and quickly found that DHCP had been broken. I have slamming my head against the wall a bit, I remembered something I had done years ago to this image to avoid an issue where PVS was failing in an older split-scope environment we had - set the PVS service "BNDevice" as a dependency of the DHCP service so that it would request the correct IP address during the hand-off to the OS (Thanks to Syxin https://www.syxin.com/tag/bndevice/).

This obviously was preventing DHCP from starting since the BNDevice service no longer existed after removing PVS tools. Simply needed to reverse that change:

HKLM\System\CurrentControlSet\Services\dhcp\DependsOnService

Remove BNDevice from the list of dependencies.

January 12, 2016

Upgrading R730 with NVIDIA K1 GRID Card

I recently ordered a couple of Dell R730 servers and then got a subsequent request to add a little graphics horsepower for our VDI environment in the form of some K1 GRID cards. Turns out, the process to add these suckers in to an existing server that wasn't specifically built out for them has a few catches - you can't just drop them in to the server and take off. It's easy to do, but there isn't much official documentation from Dell on it, so here's a quick guide:

October 23, 2015

SCEP Policy Update Troubleshooting

Because I'm a glutton for punishment, I recently started rolling out System Center Configuration Manager 2012 R2 SP1 and System Center Endpoint Protection across our VDI environment. There are always some considerations to be made in a pooled desktop / gold image type environment when loading software that uniquely identifies devices, but lucky for me SCCM/SCEP handled this just fine without any tweaking. However, there were some nuances to how SCEP policies are applied that caused some serious hair-pulling before I spotted the issues.

July 17, 2015

Outlook Credential Prompt When Opening Exchange 2013 Public Folder

After completing an Exchange 2007 > 2013 migration recently, I was left with one issue that was preventing us from stamping the project as a roaring success and moving on:

Outlook 2013 users were sometimes receiving a single pop-up prompt for credentials whenever they opened the Public Folder (we have only one). One. Single. Prompt.

Google was frustratingly unhelpful because searching for "outlook prompts for username and password when opening public folders" or something similar just resulted in a lot of folks who were always getting a pop-up that wouldn't go away. It was usually caused by an authentication failure of some sort.

However, we were in a different boat - Users got the prompt once when they first launched Outlook and opened their public folders, but after entering it they could continue - authentication worked. Next time they logged in to their PC, it would happen again. Not a show stopper, but it definitely generated its share of support calls.

April 08, 2015

Running vCenter 5.x with SQL 2012 AlwaysOn Availability Groups

After proudly starting the listener on our shiny new SQL 2012 AlwaysOn cluster, I was very eager to get vCenter moved off the brave little single-point-of-failure that is our current SQL server (a 2008 VM sitting in the virtual environment itself). I had done some research ahead of time and thought that AlwaysOn was at least sort-of supported by VMWare for protecting vCenter workloads. However, in my haste to play with a fancy new toy, I must have missed the plethora of blog posts indicating that either a) It's not actually supported at all, or b) Only Failover Clustering (shared storage) - not Availability Groups (non-shared storage) - are supported. And if you are about to do what many have done on the forums and suggest KB1037959 as evidence that they ARE supported, think again - that article is referencing support for running various clustered workloads on vSphere, not running your vCenter DB on clustered systems. Outside of a vague mention of AlwaysOn as a possible third party clustering solution to replace vCenter Heartbeat (e.g. "Best effort support"), I haven't been able to find anything official one way or the other.

But the AlwaysOn cluster was ready to go and if no one is going to tell me explicitly that I can't do it - well, that's basically an open invitation.

August 01, 2014

Pay Me Now, or Pay Me Later: DNS Edition



I hate most low-cost hosting providers. I've rarely have a great experience with one, but Web and DNS hosting providers are usually already established when I start working with a client, and migrating to another provider is typically not marked as a high priority project. I might change my mind on that after working with a client who used iPower and suddenly lost all public DNS resolution one day.

We called up iPower twice and spoke to two oblivious techs who spouted off nonsense and promised to call back after looking in to it further. Finally, on the third call a person finally told us why the client's SOA record wasn’t propagating (causing their domain name not to resolve anywhere, so no website, no e-mail, etc. for two days). Turns out they had failed to respond to their “domain verification” e-mail, which with most providers means nothing – they are just required by ICANN to prompt you to update it once a year, but most just don’t do anything if you ignore the e-mail. iPower took it a step further and deactivated their domain when there was no response.

Paul: So, if I understand you correctly, you all sent an e-mail to the registrant e-mail address to verify the registrant details. The e-mail address was wrong, so they didn't see it. So the domain was deactivated?
Paul: So you checked to see if you had the right e-mail address by ... sending an e-mail? And if the address was not correct, you just assumed we didn't need the domain?
Deepika B: Yes, you are right.
Paul: Fantastic, well done.
Deepika B: Thank you!


Bravo, iPower. Bravo.