With our growing customer base and usage, Securlogic just purchased another Sun Fire X2100. The shipment arrived on Monday and I have been running around town trying to loan a USB keyboard and struggling to get the box to boot off a USB disk. ESX 3i simply refuses to boot so I got it running with a copy of CentOS 5 (LiveCD) using the Fedora Live USB Creator. Lesson learnt - keep a USB keyboard or a USB to PS/2 converter near you and don’t buy a server without a DVD drive.
Meanwhile, the virtualization infrastructure is due for an upgrade (we’re still on VMware Server 1.x). I’m evaluating VMware ESX 3i, VMware Server 2.0 (on CentOS 5.2), Citrix XenServer and Sun xVM VirtualBox and I’ll just drop a few quick pointers for those who are going though the same process as I am.
- Both VMware ESX 3i and Citrix XenServer installs directly on bare metal and will only work if your server is listed in their Hardware Compatibility List (HCL).
- Both VMware ESX 3i and Citrix XenServer have small footprints that installs quickly on a server with minimal configuration. Most of the time network information was the only configuration needed.
- Both VMware ESX 3i and Citrix XenServer required an additional Windows node for installation of management utilities. This can be your desktop/laptop and shouldn’t be much of a problem to most people unless you’re a Mac guy.
- Both VMware ESX 3i and Citrix XenServer depends on your hardware RAID controller for RAID. They cannot perform software RAID and does not support FAKERAID/HOSTRAID controllers as well. (There is a workaround for Citrix but we did not test it as it is not an officially supported feature.)
- Both VMware ESX 3i and Citrix XenServer have awesome management tools that provide real-time information of memory usage, CPU usage, etc.
- oth VMware ESX 3i and Citrix XenServer could readily connect with an iSCSI SAN storage. Unfortunately, we did not have the hardware available to test how easily this could be done.
- Citrix XenServer is compatible with more “whitebox” hardware (i.e. non-branded server hardware) than VMware ESX 3i.
- VMware ESX 3i is supposedly capable of higher workloads* than VMware Server 2.0.
- Citrix XenServer is supposedly blazing fast* when used to run Linux guests.
- Citrix XenServer required a CIFS or NFS share to host ISO images for installation of guest O/S which we found very inconvenient. Isn’t virtualization supposed to reduce your hardware count?
- Citrix XenServer (being based off Xen) had to install a custom kernel into your Linux guest O/S. We did not have an AMD-VT capable processor and thus was unable to verify if this was the case for those with CPUs built with virtualization support.
- Citrix XenServer (being based off Xen) was unable to run Windows as a guest O/S since we did not have an AMD-VT capable processor.
- VMware Server 2.0 provides a web interface that allows remote management (including console access via a browser plug-in for Mozilla and IE) but this web interface is not compatible across different versions.
- VMware Server 2.0’s web interface requires a Java container (Apache Tomcat 6.x) to be installed and really sucks memory!
- Both xVM VirtualBox and VMware Server 2.0 requireed a base O/S. This required more time and effort to set up than VMware ESX 3i and Citrix XenServer.
- Both xVM VirtualBox and VMware Server 2.0 will be supported on most hardware that Linux or Windows supports except if your CPU is not in their HCL.
- xVM VirtualBox can be installed on Solaris 10 as a host O/S, allowing you to leverage the power of ZFS and possibly Solaris Zones (native performance).
- xVM VirtualBox requires development packages to be installed in RHEL5. (We didn’t try installing it on Solaris yet).
- xVM VirtualBox does not have the remote management capabilities of VMware ESX 3i or VMware Server 2.0. It works more like a client app and requires that your server runs a GUI (X11), somewhat like VMware Player.
Verdict? VMware Server 2.0 fits our requirements best. Here’s why:
- No FAKERAID/HOSTRAID support. We need software RAID. Software RAID gives us control, and allows easy monitoring and data recovery. If you have a goofy FAKERAID/HOSTRAID controller like the nVidia MCP series on the Sun Fire X2100, you are usually better off with software RAID.
- Data recovery. I am skeptical about data recovery on a failed VMware ESX 3i or Citrix XenServer. Since the disks are using proprietary file systems, data recovery can potentially be a big problem.
- Remote management. The ability to manage from the web is a plus, although we had instances of the browser console plug-in failing across different VMware Server 2.0 builds, we hope this issue gets ironed out soon. The lack of any proper form of network-based management simply kicks Sun xVM VirtualBox out of the league. (VNC? No thanks.)
On a side note, we love the management tools provided by Citrix XenServer and VMware ESX 3i. If not for the lack of hardware compatibility, we would have probably gone with VMware ESX 3i and Citrix XenServer second in the league. The preference for VMware ESX 3i goes towards the ease of installation/deployment of a VM. Citrix XenServer required an additional CIFS/NFS/FTP/HTTP server for installation which was not a resource we had readily available for production use at the datacenter.
In conclusion, all four products have their own strengths. You should consider them carefully and decide which suits your environment best.
Edited on March 24, 2009: Added Citrix XenServer and more reasons why we choose VMware Server 2.0.
Justin Lee is a freelance Web 2.0 and Systems Consultant for Securlogic Singapore and currently works closely with core ISP engineering teams in Singapore during his day job.
hi,
in regards to this below,
” xVM VirtualBox does not have the remote management capabilities of VMware ESX 3i or VMware Server 2.0. It works more like a client app and requires that your server runs a GUI (X11), somewhat like VMware Player. ”
Ever heard of VBoxHeadless ?? it isa tool that comes with Virtual Box that allows you to connect to the virtual machines over a remote desktop connection, so there’s no need for the VirtualBox GUI.
Link : http://www.howtoforge.com/vboxheadless-running-virtual-machines-with-virtualbox-2.0-on-a-headless-ubuntu-8.04-server
Best Regards
It’s not the console management tool I’m referring to. It’s the ability to fully manage the VM (add disk, change RAM settings, allocate CPU resources, migrate to another host, backup, etc.) from a single tool. VirtualBox lacks this ability.
With respect to Sun’s VirtualBox I think you’re comparing apples to oranges when put up against Xenserver and VMware ESXi. Xenserver and ESXi are bare-metal Type I hypervisors while VirtualBox is a Type II hosted hypervisor. VMware Server is also a Type II hypervisor but features the remote web-based management of a Type I hypervisor. VirtualBox should be compared to VMware Workstation and Microsoft VirtualPC or Parallels Workstation to make any comparison meaningful and fair.
We use VMWare server just about everywhere because it works really well on commodity hardware. We use inexpensive refurb optiplex machines with a beefy NAS over NFS. The biggest reason we like VMWare server is because we have control over the base operating system which is 90% Ubuntu server and 10% CentOS 5. We use scripts to shutdown the VMs, copy the vm files to the NAS and start the VMs backup. On a stand-alone machine without a NAS, the same scripts copy the vm files to esata or usb hard drives. A lot of our scripting is centralized from a linux script host that uses rsa certificates to ssh to the vmware hosts. That way we can centralize our backup schedules and log files are copied to a single repository where we use Splunk to monitor for failure conditions.
None of that is possible in the other offerings. However, XenServer 5.5.0 is a VERY VERY close second as it has a documented and supported base operating system loosely based on CentOS 5 that we are currently testing our scripts on.
Thanks for the post. I found it very informative.
I can only chime in on my experience with VMware server 2.0 and VirtualBox but I’m getting ready to test VMW 2.0, ESXi, and VirtualBox for a multi-site server virtualization scheme proposal to the company I’m consulting for.
I have set up virtual hosts before with VM Server 1.02/2.0 (Quad - dual core opterons, 32Gb RAM, 9Tb iSCSI SAN), but my comparative experiences with both are on a Dual core AMD64 4800+, 4Gb DDR2, 750Gb SATA2, ASUS A8N-SLI MB.
VMware Server (and ESXi) have the advantages of good industry recognition, hardware built with it in mind, and it’s so “popular” its name is almost layman recognizable making it easier to “sell”.
VMware detracting factors are the web only console (slow and twitchy), that the freeware demo is speed crippled vs. ESX(i), that ESXi is not console manageable, and the Windows only administration client is HUGE (although i haven’t tried it yet as I can’t see shelling out the cash for the rather slim “approved hardware” list to build a server from). The speed crippling of the freeware versions is the biggest detractor as it makes production emulation testing cost prohibitive, or very slow.
VirtualBox (which I have been using for approx. 3 months) has a small disk and resources impact, is snappy quick (compared to VMware), and can import a variety of VM disk types (vmware migration caveat that the disk image must be a monolithic .vmdk and not chopped into 2Gb chunks. If it is in 2Gb .vmdk chunks you must convert them into 1 file via VMware 1st before importing into VB).Once imported the guest OS’ run fine.
VB’s detractor situations are not quite enterprise level in features (yet), no remote virtual machine management app (although VNC is acceptable), and it seems to grab host memory and not give back all of it (this needs more thorough testing).
I’ll have more definite data and experiences by mid-summer, but right now I’m leaning towards VB with a custom kernel on the host.
The versatility of a type II vs. a type I host is very appealing.