Struggling with a Linux server that replies on the wrong network interface? We’ll break down the common cause of asymmetric routing and give you three real-world solutions, from a quick kernel tweak to a permanent policy-based routing fix.

That Infuriating Linux Routing Puzzle: When Your Server Replies on the Wrong Interface

I’ll never forget the 3 AM call. Our primary database, prod-db-01, was flapping. It was online, then offline. Replication was failing, then catching up. We checked everything: hardware, the database service, logs… nothing. It took us two hours of frantic troubleshooting to realize the server was accepting replication traffic on its dedicated 10.10.20.0/24 data NIC but replying out of the 192.168.1.0/24 management NIC. The stateful firewall, doing its job, saw this nonsensical asymmetric traffic and dropped the reply packets into the void. This is one of those “rite of passage” problems every engineer hits, and it’s a hair-pulling experience until you understand what’s really going on.

So, Why Is This Happening? The Single Routing Brain

The root of this problem is simple but not obvious. By default, a Linux kernel uses one main routing table to make all its decisions about where to send outbound traffic. When a packet needs to go to a network that isn’t directly connected, the kernel looks at this single table, finds the default route (the “gateway of last resort”), and shoves the packet out that interface. Period.

It doesn’t care which interface the original packet arrived on. It gets a request on Interface A, processes it, and when it’s time to reply, it just asks the main routing table, “Where do I send this?” The table usually points to the default gateway on Interface B. The result is asymmetric routing:

• Ingress: Client → Firewall → Server on `eth1` (10.10.20.50)
• Egress: Server on `eth0` (192.168.1.50) → Firewall → Client

Any stateful firewall worth its salt will see the reply coming from a completely different IP than the one the request was sent to and correctly identify it as suspicious, dropping the packet. This leads to connection timeouts and intermittent failures that are a nightmare to diagnose.

The Solutions: From a Band-Aid to Brain Surgery

Here are three ways to deal with this, ranging from the quick-and-dirty to the architecturally sound.

1. The Quick Fix: Loosening the Kernel’s Rules

The Linux kernel has a feature called Reverse Path Filtering (`rp_filter`). In its default “strict” mode (`net.ipv4.conf.all.rp_filter = 1`), it drops incoming packets if the route back to the source IP wouldn’t use the same interface the packet came in on. It’s a great security feature to prevent IP spoofing.

You can tell the kernel to be less strict, which can “fix” the symptoms of asymmetric routing by allowing the weird traffic patterns. This is a hack, not a real solution.

You can temporarily set it to “loose” mode:

sysctl -w net.ipv4.conf.all.rp_filter=2

To make it permanent, add this to /etc/sysctl.conf and reboot or run sysctl -p:

net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2

Warning: I almost never recommend this. You’re not fixing the routing; you’re just telling the kernel and your firewalls to ignore bad routing. It can mask deeper network issues and reduce your security posture. Use this only for a temporary diagnostic or if you have a very specific legacy reason.

2. The Permanent Fix: Source-Based Policy Routing

This is the right way to solve the problem. We are going to give our server’s kernel a bigger brain. Instead of one main routing table, we’ll create multiple tables and use rules to tell the kernel which table to use based on the packet’s source IP address.

Let’s assume the following setup:

Here’s the game plan:

1. Name your new routing tables. We’ll add two new tables, one for management and one for data, to make it easy to reference them. Edit /etc/iproute2/rt_tables and add these lines at the bottom:

   # Custom routing tables
   100 T100
   200 T200

2. Populate the new tables with routes. For each table, we’ll add a route for the local network and a default route via the correct gateway.

   # Configure table T100 (for management traffic)
   ip route add 192.168.1.0/24 dev eth0 src 192.168.1.50 table T100
   ip route add default via 192.168.1.1 dev eth0 table T100
   
   # Configure table T200 (for data traffic)
   ip route add 10.10.20.0/24 dev eth1 src 10.10.20.50 table T200
   ip route add default via 10.10.20.1 dev eth1 table T200

3. Create the rules. This is the magic. We tell the kernel: “If a packet is coming FROM this IP, use this specific routing table.”

   # If packet is from 192.168.1.50, use table T100
   ip rule add from 192.168.1.50/32 table T100 priority 100
   
   # If packet is from 10.10.20.50, use table T200
   ip rule add from 10.10.20.50/32 table T200 priority 200

Now, when your server replies to traffic that came in on eth1, its reply will have a source of 10.10.20.50. The new rule will catch it and force the lookup into table T200, which correctly sends it out via the 10.10.20.1 gateway. The symmetric routing path is restored, and your firewall is happy.

Pro Tip: These rules and routes will disappear on reboot. To make them permanent, you need to add them to your network configuration. For Debian/Ubuntu, this often involves using post-up commands in /etc/network/interfaces.d/. For RHEL/CentOS, you would create rule-ifname and route-ifname files in /etc/sysconfig/network-scripts/.

3. The ‘Nuclear’ Option: Rethink Your Architecture

As a Lead Architect, I have to say it: sometimes the best fix is to take a step back and admit the design might be the problem. Does this server really need two interfaces with two different default gateways?

Ask yourself these questions:

• Is this for throughput? If so, would NIC bonding (LACP) be a better solution? It provides redundancy and increased bandwidth while presenting a single IP to the OS.
• Is this for network segmentation? Great! But maybe only one of those networks needs a *default gateway*. The management network might only need to talk to other servers in the 192.168.1.0/24 subnet, which it can do with a simple static route, eliminating the need for a second default gateway and this whole problem.
• Can you use VLANs? Instead of two physical NICs, you could have one NIC trunking multiple VLANs, which can simplify the physical layout, but you still need to get the routing right.

Often, the need for multiple default gateways is a sign that network segments are not as isolated as they should be. The cleanest solution is a single default route for primary traffic and specific, static routes for any other isolated networks the server needs to reach. It avoids the complexity of policy-based routing entirely.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions