vSphere Upgrade Saga: Migrating Existing Hosts to vSphere 5, Host Profiles not Enough!

This is the week end I move all my existing hosts to vSphere 5, to make way for some other work. Actually, I will migrate 2 of 3 hosts, while leaving one host at vSphere 4, more later on why. While Host Profiles has greatly improved, it is still not sufficient when updating and migrating nodes that use physical virtual NIC devices such as HP Flex-10. Why? Because they want to know the MAC Address when you apply the profile. This is a chicken and the egg sort of approach to Host Profiles. Continue reading “vSphere Upgrade Saga: Migrating Existing Hosts to vSphere 5, Host Profiles not Enough!”

vSphere Upgrade Saga: Adding New Host Crashes vCenter

As you know from my previous post, I just added a new blade to my environment. But everything was not as clean as I desired. For some reason when I went to move a existing node from one vSphere Cluster to another, vCenter crashed. I thought it was the state in which I left the node, but alas it was not. So what would make vCenter Crash, VMware Update Manager stop showing up as a plugin, and other management tools suddenly stop working, when they worked fine before? Continue reading “vSphere Upgrade Saga: Adding New Host Crashes vCenter”

vSphere Upgrade Saga: New Blade, More Memory, More Problems

I recently purchased my 4th blade and enough memory to bring 3 of my nodes to 3x what they had before. However, this upgrade was not without its issues. Memory upgrades were no issue, but adding the 4th blade to my c3000 was a pretty significant issue. Continue reading “vSphere Upgrade Saga: New Blade, More Memory, More Problems”

vSphere Upgrade Saga: It is All About EVC

I was a little confused about EVC recently, and hopefully this will clear it up for many others. My vSphere 5 Upgrade has been halted due to work, family, and my travel schedule. I wanted to put out one more staging tip for vSphere 5 with respect to EVC. This will come in handy later as I add in a new blade with a pair of Intel Westmere chips so I can play with AES&I as well as the new Intel TXT functionality.  Continue reading “vSphere Upgrade Saga: It is All About EVC”

vSphere Upgrade Saga: Staging Upgrade to vSphere 5

I have done all my homework, planned my upgrade to vSphere 5, ensured my licenses are available and all that, so now it is time to stage the upgrade using my spare node. Why stage the upgrade and not perform it? By staging using my spare node, I am able to test to see if firmware and other upgrades work appropriately. This is crucial to the success of the upgrade of the other nodes. In essence, all other subsystems are upgraded before I hit the production nodes, which means vCenter is also upgraded but first I backup my databases, etc. As well as the VM running vCenter.

Here are the steps I followed:

Step 0: Review any updates to firmware for my HP BladeSystem c3000 just in case something new came out since the last time I looked. There was a new upgrade for the Onboard Administrator.

Step 1: Given this upgrade was available the first step was to upgrade the Onboard Administrator firmware, which I could do directly talking to the Onboard Administrator web interface. Quite simple, and easy. Unlike previous Onboard Administrator ungrades there was no issues with loosing power or connectivity to any of the blades. While doing this, I made out a shopping list to ensure even more redundancy: Another Onboard Administrator as well as a replacement Insight Display Module.

Step 2: Upgrade the Flex10 Virtual Connect firmware. This is required to use vSphere 5. You must upgrade any Flex10/Virtual connect firmware to version 3.30 in order to use vSphere 5. Step 2 was to upgrade the secondary Flex10 first using the command line Virtual Connect Support Utility (VCSU) using the update command.

Step 3: Upgrade the primary Flex10 Virtual Connect firmware. There was an oddness with this upgrade using the VCSU, and that was the VM running the VCSU lost connectivity which seemed to cause a reboot. So something inside the VCSU failed and rebooted, which is really a horrible failure mechanism. However, I only lost network connectivity for a few seconds. Which was also much better than the last Flex10 Upgrade I did.

Step 4: Upgrade the firmware within my spare blade using the HP Firmware DVD v9.30. There was an ILO update, so to be sure all firmware was upgraded, I booted from the DVD twice just to be sure I caught everything.

Step 5: Install vSphere 5 ESXi onto the spare blade just upgraded. The trick is to ensure you pick the proper disk, in this case local disk instead of a SAN drive. I double checked this before proceeding. The install is so much easier than that of ESX.

Step 6: Configure the Management Network for the spare blade to not use DHCP but to use the provided IP Address, Netmask, and hostname.

Step 7: Decided to upgrade my existing vCenter instead of using the Linux appliance, mainly because I already have a Windows configuration for vCenter and the Database server.

Step 8: Backup your vCenter Database and SSL certificates. vCenter 5 will remind you of this as well and nicely ask you to ensure you are happy with your backup.

Step 9: Perform vCenter Upgrade

Step 10: Perform vCenter Client Upgrade

Step 11: Install vSphere Web Client for new License Reporting functionality. This required me to find out which process held port 9443 open, which turned out to be something running Java. However, what I do not know. There was a Java update to be made, once I did that, this error went a way. So I went through and removed all my third party tools from the vCenter server which eventually fixed the issue. A part of this step was to register your vCenter Server with the Web Client Server so that it can communicate.

Step 12: Upgrade vCenter Update Manager. Once more this is a polite reminder to make a backup first of your VUM database which requires you to select the I am happy with my backup checkbox. I find that a great improvement. While VUM upgraded fine, it did say it wanted to reboot my host, eventually that will happen as a matter of course. Need to test if everything comes back up on boot.

Step 13: Install all the vCenter Support Tools, Dump Collector, Syslog Collector, and Auto Deploy but not the Authentication Proxy. These are all new for vCenter 5. The key to these installs, is that if it asks, yes you want the Integrated VMware vCenter Server Installation. The Authentication proxy requires IIS which could cause issues so we hold off on that for now. Now reboot your vCenter server and make sure everything runs.

Step 14: Reconnect the ESX hosts to vCenter which deployed the new vCenter Agents onto the hosts.

Step 15: Create a new cluster so that I can enable EVC mode for Nahalem or better processors. This will help with new blades once they arrive. Somehow EVC was disabled on my other nodes.

Step 16: Configure VUM and import the vSphere 5 ESXi image for later use. The VUM the client for some reason did not enable by default which took going to VMware’s KB Article 1015223 to solve.

Step 17: Configure the newly installed spare node using the old nodes Host Profile. If you have Enterprise Plus licenses the use of Host Profiles for this type of staging is incredibly useful. Even though I am moving from ESX to ESXi, use of Host Profiles to at least get the network configuration correct is very important.  If you do not have Enterprise Plus, you can use an Evaluation Mode to do this. The gotcha here is that I am moving from ESX to ESXi so some of the Host Profile elements will not apply. So not only do I do a clone of the Host Profile, apply that profile, but I need to delete and recreate the Host Profile once the spare node is fully configured else there will be compliance issues.

Step 18: Reboot Node and enter the BIOS to set HP Power Profile to Custom and HP Power Regulator to OS Control Mode per VMware KB Article 1018206. Save and reboot. You can do this as a part of any of the node upgrade steps previously mentioned, but without this you will not be able to take advantage of the power controls within vSphere 5.

Step 19: Upgrade vSphere 5. This is a good test of your VUM install from Step 16. As of this writing there are at least 2 patches for vSphere 5. One requires a reboot.

At the moment, the system is staged for migration of VMs and upgrade of the other nodes. Staging such an upgrade is important as now we know that the firmware upgrades work, I can enable the proper EVC mode, and the other systems are still manageable.

WordPress: Optimizing Steps

As you know, I have been working with my WordPress installation to improve security, but also to improve site optimization and speeds to download, etc. In this tools such as Google Page Speed and Yahoo YSlow have been incredibly useful. My goal has been to increase the Page Speed and Yahoo YSlow scores to as high as possible. This takes quite a bit of effort to achieve, I had some failed starts, bad plugins, and the need to code things to optimize the site more. So here is how I achieved my current score of 82/100 with Pagespeed. More work will be done with the ultimate goal of achieving 100/100, but that will take quite a bit of coding it appears. Continue reading “WordPress: Optimizing Steps”