A customer hardcoded an IP address for your service, and now their app is broken after your migration. Here’s a senior DevOps perspective on why this happens and how to fix it, from the quick hack to the permanent architectural solution.

They Hardcoded an IP Address, Didn’t They? A DevOps Guide to Surviving Shadow IT

I still remember “Project Chimera.” It was an internal dashboard the Sales team built over a weekend using some low-code platform. They were so proud. It pulled data directly from a reporting replica of our main production database, prod-reporting-db-01. The IP was right there in their config panel, easy as pie. They demoed it to the CTO, got a round of applause, and we in engineering didn’t hear a peep. Six months later, we did a routine database migration. We gracefully failed over, updated all our service DNS records, and went home. The next morning, my phone was on fire. Sales was dead in the water. Their precious dashboard was throwing a fit. After an hour of frantic debugging, we found it: they had hardcoded the old replica’s IP address. They didn’t see a database; they saw 34.201.54.112, a magic box that gave them numbers. And that, right there, is a story I see play out, time and time again.

The “Why”: More Than Just a Bad IP Address

When someone hardcodes an IP, it’s not just a technical mistake. It’s a symptom of a communication breakdown. They see a system that works and don’t understand the complex, dynamic environment underneath. To them, an IP is a permanent address, like a house. To us, it’s more like a hotel room number—subject to change at any moment due to scaling, migrations, failovers, or blue/green deployments.

The core problem is treating a dynamic service endpoint as a static, unchangeable entity. Modern infrastructure is cattle, not pets. We terminate instances, shift traffic, and re-route requests constantly. Relying on a specific IP is building your house on a foundation of sand. It was never a matter of if it would break, but when.

The Triage: Getting Them Back Online

Okay, so the client is down, and they’re blaming you. Your first job is to stop the bleeding. Here are three ways to approach the problem, from the immediate band-aid to the long-term cure.

1. The Quick Fix: The “Get Off My Phone” Reverse Proxy

This is the dirty, fast solution to get them working while you coordinate a proper fix. The goal is to intercept traffic going to the old, dead IP address and forward it to the new, correct service endpoint. You’re basically creating a forwarding address for their bad assumption.

Let’s say they hardcoded the IP of your old load balancer, 18.212.99.45. Your new service lives behind a load balancer with a DNS name like api-v2.techresolve.com. You can spin up a temporary Nginx instance and use a simple NAT rule or a reverse proxy to redirect the traffic.

Here’s a quick-and-dirty Nginx config to do just that on a server that you can assign the old IP to:

server {
    listen 80;
    server_name 18.212.99.45; # The OLD IP they are hitting

    location / {
        # The NEW, correct DNS endpoint
        proxy_pass http://api-v2.techresolve.com; 
        
        # Pass along the original headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Warning: This is a temporary hack. It introduces a new single point of failure and adds latency. Create a high-priority ticket to decommission this proxy the moment the client confirms they’ve updated their configuration. Do not let this become permanent infrastructure.

2. The Permanent Fix: Preach the Gospel of DNS

The quick fix bought you time. Now, you need to educate the client and get them to do the right thing. This is a communication challenge as much as a technical one.

The solution is simple: Use DNS. Always.

You need to clearly instruct their development team to stop pointing to an IP and start pointing to a hostname that you control. This hostname is your contract with them. Behind that name, you can change IPs, load balancers, and entire cloud providers, and as long as you keep the DNS record updated, their application will never know the difference.

Your email to them should be polite but firm. “For service reliability and to prevent future outages during our routine maintenance, please update your application’s endpoint configuration to use the hostname api.techresolve.com instead of the IP address. This will ensure seamless transitions during future infrastructure upgrades.”

3. The ‘Nuclear’ Option: Architectural & Process Guardrails

This problem happened once. How do you stop it, and problems like it, from ever happening again? You need to make it difficult, or impossible, to do the wrong thing.

This is about building organizational and architectural guardrails:

• Implement an API Gateway: Services like AWS API Gateway, Kong, or Apigee become the single, managed front door for all your services. You give clients an API key and a stable gateway URL. This abstracts your backend infrastructure completely and gives you rate limiting, authentication, and monitoring in one place.
• Provide Client SDKs: If you have the resources, build a simple client library (e.g., a Python package or npm module) for your customers. The SDK can handle endpoint discovery, authentication, and retries under the hood. They just call techresolve.getData(), and your library does the right thing. You’ve made the easy path the correct path.
• Better Onboarding & Documentation: Your “Getting Started” guide should have a big, bold, red box that says, “DO NOT CACHE OR HARDCODE THE IP ADDRESS OF THIS HOSTNAME.” Make it explicit. Assume your users will take the fastest shortcut possible unless you tell them not to.
• Deprecate Direct IP Access: A more aggressive move is to use firewall rules or security groups to block direct access to your application servers. Force all traffic through your load balancers or API gateway, which are only accessible via their DNS names. This makes hardcoding an IP address useless from the start.

Ultimately, the hardcoded IP is a teachable moment. It’s a chance to improve not just a client’s configuration, but your own processes, documentation, and architecture to build a more resilient and predictable system for everyone.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions