SEO Summary: A VPN over a direct datacenter cross-connect adds needless complexity and failure points. We explore why this anti-pattern exists and provide robust alternatives, from proper network segmentation to a modern service mesh approach.

Stop Putting a VPN on Your Cross-Connect: A Senior Engineer’s Take

It was 2 AM on a Saturday. PagerDuty was screaming about a full-system outage. The monitoring dashboards were a sea of red—our application tier couldn’t reach the primary database cluster. The weird part? The network graphs showed the cross-connect between our two datacenter cages was up. The switches were fine. After 45 minutes of frantic debugging, we found it: an expired OpenVPN certificate. A certificate for a tunnel running over a dedicated, physical fiber cable that ran exactly 50 feet from one of our racks to another, within the same secure building. We were down for an hour because we put a virtual lock on a physical door that was already inside a bank vault. It was security theater at its most destructive, and I promised myself, never again.

The “Why”: Security Policy vs. Physical Reality

I see this question pop up on forums and Reddit, and that 2 AM incident always comes to mind. “Would you use a VPN for a datacenter cross-connect?” My gut reaction is a hard “No,” but I get why juniors ask. It usually stems from a well-intentioned but misguided security policy: “All traffic between systems must be encrypted.”

On paper, that sounds great. In reality, a datacenter cross-connect is a private, dedicated, physical link that you pay a premium for. It’s not the public internet. It’s a glorified, really, really long Ethernet cable that you control at both ends. Adding a VPN on top of this introduces:

• A Second Point of Failure: The VPN software, its configuration, and its credentials (like that expired cert) can fail, even when the physical link is perfect.
• Performance Overhead: You’re adding encryption/decryption latency and CPU cycles for no real security gain against a realistic threat model. Who is going to physically tap a fiber cable inside a secure Equinix facility?
• MTU Black Holes: VPNs encapsulate packets, which can lead to nasty, hard-to-diagnose MTU size issues and packet fragmentation if not configured perfectly.

The root cause isn’t a technical problem; it’s a communication problem. It’s a security policy applied without context. So, how do we fix it properly? Here are the approaches I’ve used, from the “get it working now” hack to the architecturally sound solution.

The Solutions: From Duct Tape to Zero Trust

Solution 1: The “It’s 3 AM” Fix – The SSH Tunnel

Let’s be real. The system is down, your boss is calling, and you don’t have time to re-architect the network. You need to get traffic flowing securely, right now, to bypass that broken VPN. This is the duct tape you use to get everyone back to sleep.

You can establish a simple, reliable SSH tunnel from an application server to the database server. It’s encrypted, easy to set up, and bypasses the problematic VPN layer entirely. For example, to forward local port 3306 on app-prod-web-01 to the database server prod-db-01 on port 3306:

# Run this on app-prod-web-01
# It forwards local port 3306 to prod-db-01's port 3306
# The -fN options put it in the background
ssh -fN -L 3306:prod-db-01:3306 user@prod-db-01

Your application on app-prod-web-01 can now connect to localhost:3306, and the traffic will be securely tunneled to the database. Is it permanent? Absolutely not. It’s a temporary fix that creates its own technical debt. But it will save your bacon in an emergency.

Solution 2: The “Do It Right” Fix – Network Segmentation & ACLs

This is the real answer. This is what that expensive cross-connect was built for. Instead of treating the link like the untrusted internet, you treat it as an extension of your private network and secure it at the network layer (Layer 2/3).

The Plan:

1. Create a Dedicated VLAN: Work with your network team. Ask them to configure a dedicated, non-routable VLAN (e.g., VLAN 400) that only exists on the specific switch ports connected by the cross-connect.
2. Use Private IP Space: Assign IPs from a private RFC1918 range (like 10.40.0.0/29) to the interfaces on each side of the link. These IPs should only be for this connection.
3. Apply Firewall Rules (ACLs): On the switches or routers at each end, apply strict Access Control Lists. If app-prod-web-01 (10.40.0.2) only needs to talk to prod-db-01 (10.40.0.3) on port 3306, then that is the only rule you should have. Deny everything else.

Pro Tip: This approach is fast, extremely reliable, and leverages the hardware you’re already paying for. It correctly defines the security boundary at the edge of the link, not on every individual server. It respects the physical reality of the connection.

Solution 3: The “Zero Trust” Fix – Application-Layer mTLS

Sometimes, you get pushback. The security team insists, “We don’t trust the network. Any network. Ever.” Or maybe you’re building a modern, cloud-native platform and want to abstract the network away entirely. In that case, you move encryption up the stack from the network layer to the application layer.

Enter the service mesh with mutual TLS (mTLS). Tools like Istio or Linkerd create a “mesh” of proxies that sit alongside your applications. All service-to-service communication is automatically and transparently encrypted and authenticated, directly between the workloads themselves.

With this model, you don’t care if the underlying network is a cross-connect, a VPN, or the public internet. The communication from app-prod-web-01‘s proxy to prod-db-01‘s proxy is encrypted with its own short-lived certificates. The network just becomes a dumb pipe for encrypted packets.

This is the “nuclear” option because it’s a significant architectural decision, but it’s also the most future-proof. It enforces a “Zero Trust” security model where you assume the network is hostile, and it completely satisfies the “encrypt everything” policy without the pitfalls of a network-level VPN.

Conclusion: Choose The Right Tool For The Job

So, which should you choose? Here’s how I break it down for my team:

That 2 AM outage taught me a valuable lesson: don’t solve a policy problem with the wrong technology. A cross-connect is a trusted physical path. Secure it at its boundaries with proper networking, or leapfrog the problem entirely with a modern, application-aware approach like a service mesh. But please, for the sake of your on-call engineers, stop putting a VPN on it.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions