Ok, today, I discovered I am still an idiot.
Yep, I tried to add 2 public networks to one of my CloudSigma servers and one of them didn’t work.
I thought everything was to blame but my configuration (as always). Well, I managed to discover what the problem was and how to correct it.
The problem is that since there is only one default route, packets going through eth1 didn’t know how to go back to where they came from. This is solved by adding a rule and telling the kernel where to look for info on those packets:
# first my NIC configuration
## cat /etc/sysconfig/network-scripts/ifcfg-eth0
## cat /etc/sysconfig/network-scripts/ifcfg-eth1
# my routing table
## ip route
184.108.40.206/24 dev eth0 proto kernel scope link src 220.127.116.11
18.104.22.168/23 dev eth1 proto kernel scope link src 22.214.171.124
169.254.0.0/16 dev eth0 scope link metric 1002
169.254.0.0/16 dev eth1 scope link metric 1003
default via 126.96.36.199 dev eth0
# look for info on packets comming from network 188.8.131.52/23 on table 1
ip rule add from 184.108.40.206/23 tab 1 priority 500
# append to default gateway telling it to look for info on table 1
ip route add default via 220.127.116.11 dev eth1 tab 1
# flush cache
ip route flush cache
So, eth0 (18.104.22.168) is the default route. It is declared in ifcfg-eth0. If I do not declare DEFROUTE=no on eth1, then, the last NIC to become available becomes the default route. So, I specify which is the default so I can add rules later.
Then, there is eth1 (22.214.171.124) which is a completely different network. We add the rules needed for the info of it to be found on it’s own table and we add it to the default.
This works ipso facto. I don’t know if it will survive a reboot, but, hey, I know my readers will tell me if it does or not.