Virtualization Performance Testing

Virtualization technologies (such as VMware, Parallels, Xen, KVM, Waratek, etc.) have become very popular for good reasons, including but not limited to the following:

• Application density can be increased
• Server resource usage can be maximized
• Hardware costs can be reduced many-fold
• The number of servers required to run the same load can be reduced
• Applications become more portable

It is critical to do performance, load, and stress testing of a virtualization solution before rolling it out for several reasons:

• Different applications behave differently in a virtualized environment and some are more suitable than others
• Performance or stability could be affected in a virtualized environment
• Virtual machine sizing can drastically affect application behavior
• Application configuration can be different when run in a virtualized environment.

This post gives a brief outline of how to do performance, load and stress testing of a virtualization solution.

1. Include online and batch test cases as well as administrative operations
2. Test bare metal (physical server) as a baseline to be compared to virtualization results
1. See how many instance of the application can be run on bare metal
2. Push bare metal up to server capacity
3. Test virtualization
1. Run VMs on the same bare metal server or an equivalent one
2. Push one VM up to capacity, then two VMs, then three VMs, etc., up to the highest density desired.
3. Capture hypervisor server resource usage metrics (cpu, memory, disk, network)
4. Verify that VM performance compares reasonably relative to bare metal
5. Verify that VM capacity compares reasonably relative to bare metal
6. Verify that VM application stability compares reasonably relative to bare metal
7. If performance or stability is not acceptable, see if application tuning is needed or virtual machine tuning is needed.
8. Include stability tests in which high load is applied for an extended period of time.
9. Perform administrative operations while system is under high load.
10. Once performance and stability are acceptable, publish the results and the tuning and configuration needed to achieve acceptable results.