It has been a while since I added to my vSphere Upgrade Saga, but everything has been running smoothly. Now it is time to upgrade to vSphere 5.5, not because I have to, but because something within SSO broke, and the fix is to use SSO for vCenter 5.5. This leads me to upgrade to vCenter 5.5. What broke? I use the self-signed certificates from VMware for my environment. I probably should not, and that will be fixed shortly, but the long and short of it is that the certificates expired, and an upgrade of vCenter caused SSO, VUM, VIN, and other critical systems to stop talking to each other.
In order to upgrade to vCenter 5.5, you either need to have up-to-date certificates or to start over from scratch. Not having a certificate authority on hand, I chose to reinstall from scratch. Below are the steps I took. Unfortunately, reinstallation lead to other issues, such as needing old SSL certificates to access my existing database. The fact that we cannot use the certificates we want is rather limiting, but at the same time, it improves overall security.
The key to the upgrade is that you must have valid, non-expired, non-revoked certificates in place.
If you can start over, and for some shops this is possible, you need first to remove all VMware products from your vCenter Server and to reinstall, including creating a new database. In some cases, this is a good time to clean up things that are pretty messy to start with, so some planning is required, whichever direction you go. I went the route of a reinstall. If, however, you cannot reinstall due to historical and needed data, then first fix your certs! I will post more on this at a later time.
Following are the steps I took:
Step 0: Make a snapshot and backup of your vCenter server.
I did not do this step, which was a major oversight on my part, as it allows you to recover data, such as old SSL certificates.
Step 1: Remove all existing VMware products installed on your vCenter server (including removing all non-deleted product directories after product removal).
You will need your old SSL certificates, so first copy and store them somewhere. This will allow you to access your old database. Without these certificates, you will need to start your database over fresh. I wanted to try a fresh database, as mine was fairly small and I had tweaked it a few times to fix some issues. So starting fresh would allow me to start with a clean database.
Step 2: Create a new database in Simple Recovery mode for vCenter.
This step was necessary, as I had removed my old SSL certificates.
Step 3: Install using Simple Install mode.
For smaller shops or those that want everything on one server, this is by far the easiest installation method for vCenter.
Step 4: Log in to the VMware vSphere Web Client as administrator@vsphere.local (the default SSO user).
Until you tell SSO to use Active Directory or some other directory server, this is the only user that can log in via either the web client or .NET client.
Step 5: Add in your licenses.
During the vCenter install, you will be asked for your vCenter license (if you do not provide it, you will be in evaluation mode). For small organizations, evaluation mode will allow the use of all the features of vSphere Enterprise Plus, such as SVMotion, Host Profiles, etc., which can be extremely useful when you are upgrading an environment to vSphere 5.5.
Step 6: Enable Active Directory authentication.
Create users, either in Active Directory or in SSO, for each aspect of your vCenter management, such as Upgrade Manager. I use Active Directory for these users. So the first step was to add an identity source into my newly installed SSO environment. Simply login to the vSphere Web Client and navigate to Administration->Configuration (Under the Sign-On and Discovery menu item) then press the green plus sign to add an identity source. This option only shows up when you login as administrator@vsphere.local user.
Step 7: Create roles for each vCenter tool (Upgrade Manager, etc).
It is very important to create a user and role for every vCenter tool. In this case, I am creating a user named vUM, which will be used by Upgrade Manager.
Then you need to create a role. Here is a role called vDump.
Finally, attach the user to the role for later use in step 8.
Step 8: Install Tool (Upgrade Manager), employing the user previously created to talk to vCenter. This is for security purposes as laid out in the hardening guide.
Repeat Steps 6 through 8 for each tool as you install:
- vSphere ESXi Dump Collector
- vSphere Syslog Collector
- vSphere Auto Deploy
- vSphere Authentication Proxy (which needs to be deployed on a completely different host from vCenter)
- Virtual Infrastructure Navigator
- vCenter Operations Manager
- VMware vShield Manager
- vCenter Log Insight
- etc.
The need to install the vSphere Authentication Proxy, for some work I was testing, was the cause of the need to upgrade to vCenter 5.5.
Step 9: Log in to vCenter .NET Client as an administrator.
Make sure that you have added the appropriate Active Directory user for access so that you will be able to log in as an administrator via the vCenter .NET Client. You will need this client to run those elements not currently supported by the web client. Which to be honest is not much, but also, the web client feels faster and is what we are used to. I usually use both web and .NET clients as needed.
Step 10: Fix dvSwitch configurations if any are in use.
May require a login to the host to determine virtual device is connected to which virtual port. Once I fixed vMotion and FT, which use a dVS, all the other vmnics became available for assignment again through the clients. I found it is easier to do this in the .NET client, as I have been using this client for years. You will have to use the migration tool within the .NET client to migrate disconnected VMs to their new distributed virtual switches. If vCenter is living on a dVS, you may wish to set up the dVS in Ephemeral mode. I usually keep vCenter and most management tools on a non-dVS to keep problems with dVSs to a minimum.
Step 11: Install vCenter Update Manager plugin.
I received an error (see following image) that you would think is related to the database user but is actually related to the user that is running the vCenter Update Manager service. VMware KB1015233 provides the solution.
Step 12: Import your host-specific VIBs into your Patch Repository.
Since I have HP Hardware, I import four bundles from the HP Agentless Management Service Bundle, HP ESXi 5.0 Management Bundle, HP Utility Bundle, and hpnmi for ESXi 5.0. In addition, I create a baseline for just these VIBs so I can maintain them across all nodes.
Step 13: Visit your cluster’s Update Manager tab, and attach and scan your baselines.
Since we just reinstalled everything, it is also time to ensure that we have the proper baselines for updates. You may want to patch everything, but now is the time to ensure that the tools work.
Step 14: Import your 5.5 ISO so you can upgrade from within VUM.
We are just importing at this time; there will be another post on upgrading to 5.5 using VUM. This is part of staging.
Step 15: Add back any customizations you may have had.
Using the customization editor, update or add back in any customizations you use for deploying virtual machines. I only had a few, so I recreated them. If you have more than that, export and import. You may need to go back to that backup made in Step 0.
Step 16: Using the Datastore Browser, add back any templates you may have.
When you add nodes into vCenter server, templates do not get added back in. Only VMs get added back in automatically. You need to access the Datastore Browser to discover any templates on the volumes used for VMs. Many times, templates are placed on other storage as well. You still need to re-import them into vCenter.
Step 17: Enable IP pool (network protocol profiles) on proper networks so that you can install vCops.
While not technically required, this is a good thing because it lets you control what IP pools are created instead of depending on vCops to create an unknown (at least to you) pool.
Step 18: Download Client Integration Plug-in.
You do this by visiting any VM and then selecting Launch Console: the Download Client Plug-in button will appear at the upper right. You can also do this from the login page if there are no errors at the top of your browser window (such as an unsupported browser, e.g. IE 11). You must close the browser AND the .NET client if you are using both to perform the install of the Client Plug-in.
Step 19: Redeploy vCops
Remember to redeploy using steps 6 through 8 when it comes time to register vCops to your vCenter as well as to set up a data collection user. Each VMware and third-party management integration to vCenter requires its own user and role for finer-grain logging and control. After you redeploy vCops, you will need to log out and back into the vSphere Web Client (and .NET Client) to see the vCops Health Tabs.
Step 20: Reconfigure or upgrade vShield Manager.
I chose to upgrade vShield Manager, but the major issue you may have is that the time zone on the vShield Manager machine must match the time zone (and time) of the machine hosting the SSO server. Since I did a simple install, this is the time of my vCenter server. If you do not keep the time and zone in sync, then the lookup service will not be connected, which will cause integration issues with the various tools. However, if you reinstall and do not upgrade, your existing deployed edges will not import, so you may wish to consider an upgrade step. The key here is to keep your SSO virtual machine and vShield Manager virtual machine in time sync. If they are not in time sync, then you cannot attach to the SSO Lookup Service, which is a crucial step.
There you have it: 20 steps to upgrade your vCenter server. A few more steps will come in different posts, as upgrading vCenter server also implies that you need to upgrade those things that use it in order to completely reconnect all of your management tools. This is the nature of vCenter. It is the center of your virtualization management, and it impacts everything around it.
