VMWare ESXi NIC teaming load-balancing issue

I have been researching and googling to clarify how VMWare NIC teaming is behaving load-balancing traffic between ESXi host and switch.

 

Here is known factors.

1. A switch is linked to ESXi host should be switch grade, but not hub. It might cause MAC flapping situation

 

2. ESXi vSphere 4.x is not support LACP, so use static bond method on a switch side.

Ex) In Cisco case : "channel-group 10 mode on" under interface configuration mode.

Ex) In Avaya case : "

 

3. LACP will be supported on ESXi vSphere 5.1(thru DVS) with below restriction

– vSphere supports only one LACP group per distributed switch and only one LACP group per host

– LACP settings do not exist in host profiles

– LACP between two nested ESXi hosts is impossible

– LACP does not support Port mirroring

– LACP only works with IP Hash load balancing and Link Status Network failover detection

 

 

How load-balancing with IP hash?

– First of all, load-balancing means spead traffic on given or avaiable NICs(Network Interface Card), but it doesn’t mean equal amount of traffic is distributed to all participated NICs. (50/50, or 33/33/33 or 25/25/25/255…..)

– By setting up "Route based on IP hash", ESXi will choose a NIC among teaming member by using XOR algorithm with source IP and destination IP.

– Traffic pattern

A single source IP to a single destination IP = Single NIC will be used all the time

A single source IP to many destination IPs = most likely single NIC will be used all the time

A many source IPs to many destination IPs = all NICs will be utilized.

 

So, if you have single VM and communicating only single destination, all traffic will be staying a NIC all the time. Doesn’t matter how many NICs are teaming together. How about traffic is exceeding maximum  bandwidth of the link? Result will be packet lost. Remember no LACP.

Any solution? with ESXi 4.x, you need to have third party solution such as Cisco 1000v or IBM 5000v to use LACP.

 

 

 

 

References :

http://blogs.vmware.com/kb/2013/03/troubleshooting-network-teaming-problems-with-ip-hash.html#.UcrQOpxj9c4

http://wahlnetwork.com/category/tech-talks/vsphere-lab/