There have been many tests published showing great performance of SAP software running on VMware vSphere in a two-tier configuration. Both the application server and database server are running on the same VM in a two-tier configuration which reflects how many customers run SAP solutions in small and mid size configurations. In larger configurations, it becomes necessary to split out the application server and database server onto separate systems. In order to demonstrate how SAP software scales in a three-tier environment running on vSphere, a test was set up with three application server VMs and one database server VM.
Working with the Dell TechCenter lab, a PowerEdge M710 blade server with VMware vSphere 4.0.0 was used to test a set of SAP VMs in a three-tier configuration. The blade server had two quad-core Intel x5570 (Nehalem) processors and 72GB of RAM. Storage was provided by three PS5000XV Dell EqualLogic iSCSI arrays.
Each of the four VMs was configured with 18GB of virtual memory and Microsoft Windows Server 2008 x64 was installed as the guest OS. Microsoft SQL Server 2008 was used as the database and enhancement pack 4 for SAP ERP 6.0 was running on the application server VMs. The database VM also had the SAP application server components installed, but no transactions were directed through it.
Using a well understood SAP transactional workload on this three-tier configuration, three series of tests were run to measure how performance scaled as resources were added to the VMs. A one second response time criteria was used to determine the number of users. The graph below shows how the response time increased as users were added to each of the three configurations tested.
 
The first test configuration with four 2vCPU VMs achieved 1386 users. This initial test configuration had 8 virtual CPUs, one for each of the 8 physical cores in the server. In order to test the performance scaling when using SMT, the second configuration used a 4vCPU configuration for the VMs, allowing all 16 logical threads of the server to be used, and achieved a 24% increase in supported users. The Intel 5500 series processors implement Hyper-Threading, a form of Simultaneous Multi-Threading (SMT) which allows for two logical threads to run on each physical core. The two socket M710 used in this testing had 8 cores and 16 logical threads. It is not recommended to use CPU affinity for SAP VMs in production, but for this high stress testing scenario it was used to assign the vCPUs for each VM to specific logical threads.
For a final test, a second server with the same configuration was added.  All four VMs kept the same configuration from the previous test with 4vCPUs, but two of the VMs were moved to the new server.  This third configuration is similar to the first, with the number of virtual CPUs being equal to the number of physical cores.  There are 16 vCPUs assigned across the four VMs and there are a total of 16 cores across the two physical servers.   Spread across two hosts, a total of approximately 2630 users could be run at under 1 second response time representing a 90% increase in performance from the first configuration.
In terms of CPU utilization the servers were heavily utilized in all of the tests at the one second response time criteria.  In the first test the core utilization as reported by esxtop averaged 85% and in the second test with all 16 logical threads being utilized it increased to 93%.  In the final test with two hosts, the core utilization was 77% and 80%.
The results show that using SAP in a three-tier configuration on vSphere allows for excellent scaling.   Additional performance was achieved by simply adding more resources to the existing VMs (in the form of more physical cores) without making any changes to the operating system, database, or application servers running inside the VMs.
