Danté's Inferno

Ubuntu 11.04 Network Manager with 2 Interfaces

Lately I’ve been using my T-Mobile Jet Stick Broadband to connect to the Internet from my current customer. The reason for this is mainly that I’m not allowed to put my laptop on their network for security reason (and HIPPA, probably).

This morning I setup the HP ILO2 service processor on a DL380 G5 that I’m rebuilding with Oracle Linux 5.2 and wanted to be able to connect to directly to the ILO2 via the physical Ethernet port in my laptop. Additionally, I needed to simultaneously connected to the Internet via the T-Mobile stick.

For some reason, no matter the order I put the interfaces in, Network-Manager would always set the preferred routing over the physical cable!. Since that is only connected to a single ILO2, without a switch or router in between, that’s pretty useless as it tries to route all IP traffic via the ILO2!

So, I decided to plug both things in, then manipulate the routing table. Hopefully, here are all the commands to share:

Initial Routing Table

Routing table with the T-Mobile plugged in and with the ethernet plugged in. You can see the entry for the T-Mobile has an Iface entry of ppp0 (point-to-point) and the HP ILO2 has the interface of eth0.

netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
10.64.64.64     0.0.0.0         255.255.255.255 UH        0 0          0 ppp0
126.10.8.0      0.0.0.0         255.255.252.0   U         0 0          0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth0
0.0.0.0         126.10.8.33     0.0.0.0         UG        0 0          0 eth0

The important thing to notice is the default gateway, Flags=UG, is 126.10.8.33, which is the IP address of the HP ILO2 device. Not what we need!

Step 1: Remove that weird 169.254.0.0 entry

sudo route del -net 169.254.0.0/16
[sudo] password for dante:
dante@dexter:~$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.64.64.64     0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
126.10.8.0      0.0.0.0         255.255.252.0   U     1      0        0 eth0
0.0.0.0         126.10.8.33     0.0.0.0         UG    0      0        0 eth0

Step 2: Now remove the UG route

sudo route del -net 0.0.0.0 netmask 0.0.0.0 gw  126.10.8.33 eth0
dante@dexter:~$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.64.64.64     0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
126.10.8.0      0.0.0.0         255.255.252.0   U     1      0        0 eth0

Step 3: Add 10.64.64.64 as the default gateway

sudo route add -net 0.0.0.0 netmask 0.0.0.0 gw 10.64.64.64 ppp0
route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.64.64.64     0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
126.10.8.0      0.0.0.0         255.255.252.0   U     1      0        0 eth0
0.0.0.0         10.64.64.64     0.0.0.0         UG    0      0        0 ppp0

Step 4: Check traceroute for internet and lan

dante@dexter:!$traceroute -n 126.10.8.33
traceroute to 126.10.8.33 (126.10.8.33), 30 hops max, 60 byte packets

dante@dexter:~$ traceroute -n 74.1.46.163
traceroute to 74.1.46.163 (74.1.46.163), 30 hops max, 60 byte packets
 1  10.168.187.42  96.535 ms  96.455 ms  96.419 ms
 2  10.168.187.45  96.386 ms  96.354 ms  96.242 ms
 3  10.168.187.68  96.204 ms  106.176 ms  106.149 ms
 4  10.168.187.49  106.124 ms  106.099 ms  106.078 ms
 5  10.170.206.5  106.058 ms  89.680 ms  89.613 ms
 6  10.170.206.10  89.580 ms  89.517 ms  89.484 ms
 7  10.161.59.13  89.454 ms  89.424 ms  99.408 ms
 8  10.176.188.194  99.377 ms  99.288 ms  89.795 ms
 9  10.176.188.150  89.728 ms  89.697 ms  89.666 ms
10  4.53.73.145  99.592 ms  99.476 ms  89.621 ms
11  * * *
12  166.90.203.2  89.443 ms  89.290 ms  89.255 ms
13  192.168.7.14  89.590 ms  89.474 ms  89.441 ms
14  74.1.46.161  99.324 ms  109.329 ms  109.300 ms
15  74.1.46.163  119.147 ms  109.635 ms  109.556 ms