Tip of the day: policy routing

One of the most powerful features in the Linux kernel is usage of multple routing tables and policy routing.  In normal routing, decisions are made simply based on the source and destination of the packet.  In certain instances, this might not be enough flexibility - we may need to make a different decision based upon what protocol we are using, for example.  Take this example to have SMTP routed differently than the rest of the traffic coming out of the box.

Good news is that the Linux routing code makes this fairly easy.  First, you have to add the table definition to /etc/iproute2/rt_tables, so that it looks something like this:

[root@centos4 ~]# cat /etc/iproute2/rt_tables

#

# reserved values

#

#255    local

#254    main

#253    default

#0      unspec

#

# local

#

#1      inr.ruhep

100     smtp

Then, you need to setup an IPTables rule to mark the traffic that you are interested in using the alternative gateway for:

iptables -t mangle -A OUTPUT -p tcp --dport 25 -j MARK --set-mark 0x2

Then you need to setup the ip routing rule and the default route in the alternative table:

ip rule add fwmark 0x2 table smtp.

ip route add default via 192.168.1.2 table smtp

Now all SMTP traffic generated locally by the box will route out the gateway 192.168.1.2.  Note, however, that a regular 'ip route' command doesn't show you what's going on here:

[root@centos4 ~]# ip route

192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.23

169.254.0.0/16 dev eth0  scope link

default via 192.168.1.1 dev eth0

Instead, we need to specify which table to look in for the route.  Remember adding the table to /etc/iproute2/rt_tables before?  That's for human conveinence in mapping the table name to a number

[root@centos4 ~]# ip route show table smtp

default via 192.168.1.2 dev eth0

You can tell which rules and tables are in use via 'ip rule list'

[root@centos4 ~]# ip rule list

0:      from all lookup local

32765:  from all fwmark 0x2 lookup smtp

32766:  from all lookup main

32767:  from all lookup default

And finally, you can see what's in the mangle table by looking at iptables:

[root@centos4 ~]# iptables -t mangle -nvL

Chain PREROUTING (policy ACCEPT 2210 packets, 172K bytes)

 pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 2210 packets, 172K bytes)

 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)

 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 1351 packets, 153K bytes)

 pkts bytes target     prot opt in     out     source               destination

    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:25 MARK set 0x2

Chain POSTROUTING (policy ACCEPT 1351 packets, 153K bytes)

 pkts bytes target     prot opt in     out     source               destination

There's a brief introduction to policy routing in RHEL4! Special thanks to merlinthp over in #rhel for pointing me in the right direction here, and to the iproute2 documentation on policyrouting.org