Tuesday, April 8, 2014

ESXi 5.5 vulnerable to OpenSSL Heartbleed bug

Today the existence of a serious bug in OpenSSL was revealed which allows an attacker to read the memory of your system including usernames, passwords, keys etc.

More information here: http://heartbleed.com

This site will check a URL for the vulnerability:  http://filippo.io/Heartbleed/ 

This command line tool also checks for the vulnerability

Our ESXi 5.5 build 1331820 servers have the vulnerability.
And fully patched ESXi 5.5  build 1623387 (as of 4/8) shows the vulnerability as well.
I've posted in the vmware forums asking about the ETA of a patch to fix this in our VI.

Update 4/9: VMWare has supplied a KB of products affected

Thursday, January 2, 2014

Team Canada Hockey Roster Map

One of my other passions (besides  virtualization, storage IOPS, cloud engineering  etc) is hockey.
Today Steve Yzerman, team Canada GM released the Team Canada Hockey roster for Sochi.
I wanted to create a visual, infographic map of the player's hometowns and found this cool geo mashup tool which is mostly intuitive and accurate.

View 2014 Team Canada Hometown map v0.3 in a full screen map

Monday, December 9, 2013

vNFS datastore for Cluster upgrades

In our pre 5.1 environment we have multiple clusters without centralized storage.
This is a problem when the cluster nodes need to be upgraded - we can not evacuate the nodes via vMotion since the VMs are on local disk.
Here is a solution: vNFS datastore provided you have adequate local disk on your hosts.

Present an NFS datastore to the cluster via a new VM using local storage.

1 - Configure a Centos/Linux VM with the local storage and share it via NFS
2 - Add the NFS datastore to the cluster nodes
3 - Storage vMotion off the VMs from the node to be upgraded one at a time (monitor the load on the NFS vm via top - the throughput is largely based on the disk IO and secondarily the network)
4 - vMotion off the VMs now they are on the NFS datastore
5 - put the host is maintenance mode and upgrade
6 - reverse the procedure to migrate the VMs back to the upgraded host and local storage

With 5.1 local to local storage vMotion and the upcoming vSAN offering this is less of an issue, but in the interim, its saved us much downtime.

Tuesday, October 22, 2013

vCenter 5.5 upgrade fails and rolls back

Upgrading our 5.1 vCenter to 5.5, we ran into this error:

"Simple Install Setup Wizard ended prematurely because of an error"

This turned out to be related to a mismatch btw the Registry VC IP address and the expected FQDN.
VMware KB2060511 provides the solution for this issue (clean up the failed upgrade dir, and modify the registry setting)

Sunday, April 14, 2013

unregistering virtual machine hangs

We were migrating 40 VMs from one Netapp filer to another and implementing IPSpaces at the same time, so we needed to power down all 40 VMs for this.
Once the VMs were migrated, we powered on the DB VMs first - no issues.
Once the DBA verified Oracle was up, I started powering on app VMs.
But they all hung on  
 "unregistering virtual machine from source host system"

Turns out this was due to 3 of the vmwaretools updates hanging - I had started these vmwaretools updates while the DBA was working on Oracle.
These were not the same App VMs I was trying to power on, but once 2 of the hanging vmwaretools updates finally timed out, some of the App VMs finished powering on.
I manually killed the remaining hung vmwaretools update using the ESXi command line then everything proceeded fine.
Hope this helps someone else since it was not obvious at the time the vmwaretools updates were preventing other VMs from powering up.