Tuesday, September 11, 2012

TIPC: Dual Bearers - Oh my!

The 10Gbe cards I have in my cluster have two interfaces (eth1 and eth2), and so far I have only made use of one of them.  TIPC has the ability to internally bond these so that they are both used to transfer data.  The benefit is mostly redundancy (more realistic if done with separate cards), and increased bandwidth.  In order to pull this off I need to change the script I am using to configure the TIPC network on the cluster, after connecting the unused interfaces (highlighted in green):




The network interfaces plugged in here are the Solareflare 10Gbe interfaces which I reported on earlier.  Configuring the new interface and adding it to the bearer list is a pretty simple edit to my bash script (run as root):


#!/bin/bash

mussh -h monet010 -h monet011 -c '
modprobe sfc
export SUFFIX=`hostname | sed 's/monet0//g'`
ifconfig eth1 192.168.3.$SUFFIX
ifconfig eth2 192.168.4.$SUFFIX
modprobe tipc
tipc-config -netid=1234 -addr=1.1.$SUFFIX -be=eth:eth1,eth:eth2
lsmod | grep tipc
/home/mark/git/tipcutils/tipc-config/tipc-config -lw=broadcast-link/100
'

Now we can confirm (on each node that the bearers are both there):


root@monet011:/home/mark# tipc-config -nt -b
Type       Lower      Upper      Port Identity              Publication Scope
0          16781322   16781322   <1.1.10:1765777409>        1765777410  cluster
           16781323   16781323   <1.1.11:3091349505>        3091349506  zone
1          1          1          <1.1.11:3091365891>        3091365892  node
Bearers:
eth:eth1
eth:eth2

I am eager to retry some of my throughput tests which previously indicated that the NIC was saturated (so I was hitting the 10 Gbe ceiling).  Stay tuned.  Will this prove tasty, or poison?

No comments:

Post a Comment


Follow Mark on GitHub