Today I visited a client who did not apply the best practice to separate vMotion traffic in it’s own isolated VLAN. Live migration from one host to another cause the physical switch to experience unicast flooding. After much investigation, the solution was quite simple: Don’t setup your management and your vMotion network traffic to be on the same subnet.
The client experienced big network problems, especially on the network parts between datacenters.
There is a lot of VMware documentation that alludes to this fact, but it’s never very direct. For example in this PDF, “4.1 ESXi configuration Guide“, under the Networking Best Practices, we see the following:
Keep the vMotion connection on a separate network devoted to vMotion. When migration with vMotion occurs, the contents of the guest operating system’s memory is transmitted over the network. You can do this either by using VLANs to segment a single physical network or separate physical networks (the latter is preferable).
Personally, I would recommend to separate all of your traffic by subnets: NFS, FT, vMotion, and HA (VMware KB 1006989).The above PDF also refers to this:
To physically separate network services and to dedicate a particular set of NICs to a specific network service, create a vSwitch for each service. If this is not possible, separate them on a single vSwitch by attaching them to port groups with different VLAN IDs. In either case, confirm with your network administrator that the networks or VLANs you choose are isolated in the rest of your environment and that no routers connect them.
most info borrowed from: http://virtualaholics.com/2012/03/vmotion-causes-unicast-flooding/