Ok, today, I discovered I am still an idiot.
Yep, I tried to add 2 public networks to one of my CloudSigma servers and one of them didn’t work.
I thought everything was to blame but my configuration (as always). Well, I managed to discover what the problem was and how to correct it.
The problem is that since there is only one default route, packets going through eth1 didn’t know how to go back to where they came from. This is solved by adding a rule and telling the kernel where to look for info on those packets:
# first my NIC configuration
## cat /etc/sysconfig/network-scripts/ifcfg-eth0
## cat /etc/sysconfig/network-scripts/ifcfg-eth1
# my routing table
## ip route
220.127.116.11/24 dev eth0 proto kernel scope link src 18.104.22.168
22.214.171.124/23 dev eth1 proto kernel scope link src 126.96.36.199
169.254.0.0/16 dev eth0 scope link metric 1002
169.254.0.0/16 dev eth1 scope link metric 1003
default via 188.8.131.52 dev eth0
# look for info on packets comming from network 184.108.40.206/23 on table 1
ip rule add from 220.127.116.11/23 tab 1 priority 500
# append to default gateway telling it to look for info on table 1
ip route add default via 18.104.22.168 dev eth1 tab 1
# flush cache
ip route flush cache
So, eth0 (22.214.171.124) is the default route. It is declared in ifcfg-eth0. If I do not declare DEFROUTE=no on eth1, then, the last NIC to become available becomes the default route. So, I specify which is the default so I can add rules later.
Then, there is eth1 (126.96.36.199) which is a completely different network. We add the rules needed for the info of it to be found on it’s own table and we add it to the default.
This works ipso facto. I don’t know if it will survive a reboot, but, hey, I know my readers will tell me if it does or not.