I never found a complete tutorial on setting up KVM networking the way I wanted. One thing that VMware has everyone beat on is simple and effective network configurations. KVM hosts can be just as good, but it won’t draw the pictures for you so it’s difficult to visualize what’s going on and troubleshoot it when things are going wrong.
This write-up should give you all the information you need to create a robust, bonded and VLAN aware “virtual switch” configuration on your KVM host. My config uses all native Linux networking constructs. It does not make use of the newer “team” method of interface aggregation and it definitely does not make use of Network Manager; as a matter of fact unless you have express need for it I suggest you uninstall Network Manager as it can cause grief in your configuration. As with all my other KVM related write-ups, this is based on EL7 type hosts, CentOS 7.0 in my case. If you wish to adapt it for other flavors of Linux this may still give you a good starting point.
Here is an approximation of what it should look like when you’re done:
In case it’s not obvious, the shaded balls are your KVM domains. When configuring your new domains you will select the “Specify shared device name” option in virt-manager and type out the bridge you want the domain connected to. Or alternatively if you’re hand crafting your domain’s XML file it will look like this:
<interface type='bridge'> <mac address='ff:ff:ff:ff:ff:ff'/> <source bridge='virbr120'/> <target dev='vnet0'/> <model type='rtl8139'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' /> </interface>
This would connect your VM to VLAN120 per my config above. Obviously many other things in this XML are domain and environment specific so don’t just try to copy and paste that and expect your machine to work, if you’re hand editing XML – know what you’re doing. Some of the other configs that you’ll need are as follows:
Cisco 3650:
sw# config t sw(config)# interface range gi0/1,gi0/2 sw(config-if-range)# switchport trunk encapsulation dot1q sw(config-if-range)# switchport trunk allowed vlan 100,110,120,200 sw(config-if-range)# switchport mode trunk sw(config-if-range)# channel-group 1 mode on sw(config-if-range)# exit sw(config)# interface po1 sw(config-if)# switchport trunk encapsulation dot1q sw(config-if)# switchport trunk allowed vlan 100,110,120,200 sw(config-if)# switchport mode trunk sw(config-if)# description "KVM Server 1 VMNetwork bonded and trunked"
On your KVM host:
/etc/modprobe.d/bond0.conf:
alias bond0 bonding
/etc/sysconfig/network-scripts/ifcfg-eth0:
DEVICE=eth0 ONBOOT=yes NM_CONTROLLED=no BOOTPROTO=none USERCTL=no MASTER=bond0 SLAVE=yes
Make eth1 or whatever your 2nd adapter look similar, obviously change the DEVICE= line
/etc/sysconfig/network-scripts/ifcfg-bond0:
DEVICE=bond0 NM_CONTROLLED=no ONBOOT=yes BOOTPROTO=none USERCTL=no BONDING_OPTS="miimon=100 mode=4 lacp_rate=1"
/etc/sysconfig/network-scripts/ifcfg-bond0.100:
DEVICE=bond0.100 ONBOOT=yes VLAN=yes BOOTPROTO=none NM_CONTROLLED=no BRIDGE=virbr100
Like the physical interfaces, you can copy/paste this for the other VLANs you want to include in your configuration, you will have to change the DEVICE= line and BRIDGE= line in each separate config file.
/etc/sysconfig/network-scripts/ifcfg-virbr100:
DEVICE=virbr100 ONBOOT=YES TYPE=Bridge DELAY=0 BOOTPROTO=none
This one is another copy/paste candidate to bridge you into any of your VLAN interfaces, this time the only line you’ll need to modify as you copy and paste is DEVICE=. If you’d like you can add an IP address, subnet mask, etc to any of the bridge interfaces and then use that to connect to your KVM server. For me I prefer to have dedicated out-of-band interfaces for management purposes so all of my bridges are without layer 3 termination.
That’s it.
