Ethernet bonding

Bonding is one of the terms used to describe multiple Ethernet cables used to form a single virtual network link. This can be done for performance or reliability.

Bonding for performance used to be common when 100baseT was the fastest network technology that was commonly available. In 1999 servers could usually sustain considerably more than 10MB/s so a single 100baseT network interface was a performance bottleneck. At that time I worked with Cisco switches and Solaris machines that had up to four 100baseT links bonded for performance.

Nowadays Gigabit Ethernet is commonly available, most laptops have Gigabit Ethernet on the motherboard. Gigabit PCI cards are as cheap as $35, and Gigabit switches can be purchased for as little as $139. Server hardware is a little more expensive, but it's still quite cheap and commonly available.

Most people don't need more than Gigabit speed, in fact most systems can not saturate a Gigabit link due to poor application design, a slow operating system, or slow disks used to provide the data. So at this time there is little speed for bonded Gigabit networking for performance.

There is still the issue of reliability. Often you want to have two ethernet cards and cables configured so that if one breaks the network won't go down.

One annoying thing about bonding in Linux (in 2.6.x kernels) is that the module has to be loaded separately for each bond interface, and the parameters for an interface can't be changed without unloading and loading the driver (very painful if you log in to the machine via ssh over the bonded interface to do sys-admin work).

The parameters I have in /etc/modprobe.conf for bonding are:

alias bond0 bonding
options bond0 mode=1 arp_interval=500 arp_ip_target=192.168.0.1

This means that if there is no traffic on the link then every 500ms an ARP request will be sent for the address 127.128.129.130 (I used the address of my router but substituted a different value for this blog entry). An ARP request for a machine on the local LAN is a request that will always be satisfied if the machine in question and the network link are working.

The idea is that you have two switches and every computer that matters has two ethernet ports. If one port stops working (broken Ethernet card, cable, or router) then the other takes over.

The special file /proc/net/bonding/bond0 can be used to view the current configuration of the bond0 device.

Below are sample configuration files for Fedora and Red Hat Enterprise Linux to configure bonding:

/etc/sysconfig/networking/devices/ifcfg-bond0:
DEVICE=bond0
IPADDR=192.168.0.2
NETMASK=255.255.255.0
NETWORK=192.168.0.0
BROADCAST=192.168.0.255
ONBOOT=yes
BOOTPROTO=static
# GATEWAY should be the IP address to ARP ping
GATEWAY=192.168.0.1
TYPE=Ethernet

/etc/sysconfig/networking/devices/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
HWADDR=xx:xx:xx:xx:xx:xx
ONBOOT=yes
SLAVE=yes
MASTER=bond0
TYPE=Ethernet

/etc/sysconfig/networking/devices/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
HWADDR=xx:xx:xx:xx:xx:xx
ONBOOT=yes
SLAVE=yes
MASTER=bond0
TYPE=Ethernet

Note that there is nothing preventing you from having more than two devices bonded together for reliability, but I doubt that you really need that.