Struggling with internal services that can’t connect using public hostnames? Learn why this “NAT Hairpinning” problem happens and explore three practical solutions, from a quick /etc/hosts hack to a permanent Split-Horizon DNS setup.

From In-House to Out-house… and Back Again? Solving the NAT Hairpinning Puzzle

It was 2:37 AM. The PagerDuty alert screamed: API_UNREACHABLE. I SSH’d into prod-app-01, ran a quick curl, and sure enough, api.techresolve.io was timing out. The funny thing? My phone, on a cellular network, could hit the API just fine. The monitoring probes from outside our network all showed green. The call was coming from inside the house. This, my friends, is the classic hairpin problem, and it’s one of the most frustrating “it works everywhere but here” issues you’ll ever face in your career.

So, What’s Actually Happening Here? The “Why”

You’ve got a perfectly good application server, let’s call it prod-app-01 with an internal IP of 10.10.1.20. It needs to talk to a database API, which is configured in your app’s config file by its public hostname, db.techresolve.io. Your DNS is working fine, so when your app server asks for the IP of db.techresolve.io, it correctly gets back the public IP, let’s say 203.0.113.50.

Here’s the breakdown of the failed trip:

1. Request: prod-app-01 (10.10.1.20) sends a packet destined for 203.0.113.50.
2. Egress: The packet hits your default gateway, which is your corporate firewall/router.
3. The Problem: The firewall looks at the packet. It sees a source IP from its internal network (10.10.1.20) trying to reach a destination IP that it manages on its external interface (203.0.113.50).
4. Denial: For security reasons, most firewalls by default will drop this packet. It’s a security feature, not a bug! Allowing traffic to leave your network only to immediately loop back in is considered suspicious behavior, often associated with reflection attacks. The firewall is essentially preventing your network from talking to itself via the public internet.

This is called a NAT Hairpinning (or NAT Loopback) problem. The traffic goes up to the firewall and needs to make a U-turn (a hairpin) to come back to another internal server, but the firewall just says “Nope.”

The Fixes: From Duct Tape to Solid Engineering

You’ve got options, ranging from “please don’t do this in production” to “this is how we build scalable systems.” Let’s walk through them.

Solution 1: The Quick & Dirty Fix (The /etc/hosts Hack)

This is the first thing every junior engineer tries, and honestly, it works in a pinch for a single dev box. You manually tell the client server to resolve the public hostname to the target’s private IP, bypassing DNS entirely for that one lookup.

On your application server (prod-app-01), you’d edit the /etc/hosts file:

# /etc/hosts

127.0.0.1   localhost
...
# Add this line to fix the API connection
10.10.2.45  db.techresolve.io

Pros: It’s fast, simple, and requires no network changes. You can fix the immediate problem in 30 seconds.
Cons: This is pure technical debt. It’s not scalable. If you have 50 app servers, you have to edit 50 files. If the private IP of prod-db-01 ever changes, everything breaks again, and nobody will remember this hack was put in place. It’s a ticking time bomb.

Pro Tip: Use this method to diagnose the problem, not to solve it long-term. If adding a hosts entry fixes it, you’ve confirmed you have a hairpinning issue. Now, go implement a real solution.

Solution 2: The Permanent Fix (Split-Horizon DNS)

This is the right way to solve this problem. The concept is simple: your DNS server gives a different answer depending on who is asking. This is also called “split-view DNS.”

• If a request for db.techresolve.io comes from the public internet, the DNS server returns the public IP: 203.0.113.50.
• If a request for db.techresolve.io comes from your internal network (e.g., the 10.10.0.0/16 range), the DNS server returns the private IP: 10.10.2.45.

This way, your internal traffic never even tries to leave the network. It’s resolved to the correct private IP from the start. Most modern DNS systems support this. In AWS, this is trivial to set up with Route 53 using a Private Hosted Zone associated with your VPC. For on-prem, you can configure BIND with different “views.”

Pros: It’s scalable, manageable, and the industry-standard solution. It’s the “set it and forget it” fix.
Cons: It requires you to have control over your DNS infrastructure and a bit more setup time than the hosts file hack.

Solution 3: The ‘Nuclear’ Option (Enable NAT Hairpinning on the Firewall)

The final option is to lean into the problem and configure your network hardware to allow this traffic pattern. You’re asking the firewall to be smart enough to recognize this U-turn scenario and rewrite the packet to stay within the internal network.

To do this, you (or more likely, your network security team) would create a specific rule on the firewall that says: “If a packet comes from the internal network (10.10.0.0/16) and is destined for public IP 203.0.113.50, don’t send it out to the internet. Instead, forward it directly to the internal machine at 10.10.2.45.”

Warning: This can be complex and sometimes has unintended security consequences if the rule is too broad. Many network admins are reluctant to enable it. Not all hardware supports it well. Always approach this as a conversation with your networking team, not a demand.

Which One Should You Choose?

Here’s my personal take, summarized in a handy table.

Understanding why your packets are getting dropped is half the battle. Now you have a playbook. Stop patching with hosts files, start thinking about your DNS strategy, and you’ll save your future self from another 2 AM wakeup call.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions