Stop treating your network like a fragile artifact. This post dives into why the ‘set it and forget it’ approach to L3 routing is a recipe for disaster and offers real-world strategies for making network changes safe, repeatable, and boring.

Touching BGP Gives Me Anxiety: A Guide to Taming L3 Routing Changes

I still remember the 2 AM page. A junior engineer, let’s call him Alex, was on call. The ticket was simple: “Add new customer subnet to BGP advertisement.” A five-minute job. But forty-five minutes later, half our services were flapping and my phone was melting. I got on the bridge call and Alex sounded panicked. He’d tried to update a route-map on edge-router-01.sjc, but in the process, he’d momentarily wiped the existing prefix-list. For about 90 seconds, we stopped advertising all our prefixes to a major transit provider. The BGP sessions reset, routes reconverged, and chaos ensued. Alex followed the old runbook, but the old runbook was a landmine. This wasn’t his fault; it was ours. We had created a culture where touching the core network was a terrifying, rare event, which only made it more dangerous when we actually had to.

The Real Problem: Fear is a Terrible Change Management Strategy

That Reddit thread hits home because so many of us have been there. The core issue isn’t that BGP, OSPF, or EIGRP are inherently fragile. The problem is our relationship with them. We treat our core routers like priceless, hand-blown glass vases. We’re so afraid of breaking them that we only make changes when absolutely forced to. This leads to a vicious cycle:

• Infrequent Changes: We wait months, batching up dozens of unrelated changes into one terrifying “maintenance window.”
• Increased Risk: A huge, multi-part change is exponentially more likely to fail than a small, atomic one. It’s impossible to isolate the root cause if something goes wrong.
• Knowledge Decay: If you only touch your routing policy once a year, do you really remember all the intricate details and unspoken dependencies? Of course not.
• No Automation: The process is manual because “it’s too risky to automate.” This manual work, done by a nervous engineer at 2 AM, is the single greatest source of risk.

The goal isn’t to make changes less often. The goal is to get so good at it that you can do it on a Tuesday morning with your coffee in hand, without breaking a sweat. Here’s how we get there.

Solution 1: The ‘White Knuckle’ Method (A Better Manual Process)

Look, I get it. You can’t just spin up a full CI/CD pipeline for your network tomorrow. If you have to make a manual change right now, you can still drastically reduce your risk by treating it like a surgical procedure. We call this “The Ritual.”

The Ritual Checklist:

1. Write a MOP (Method of Procedure): Don’t just wing it. Write down the *exact* commands you will run, in order. Include verification commands before and after.
2. Write the Rollback Plan: What are the *exact* commands to undo your change? Have them ready to paste. If your change is to modify a prefix list, the rollback is to revert it to the old version.
3. Peer Review: Have another engineer—ideally a senior—read your MOP and rollback plan. A second pair of eyes is your best defense against a simple, catastrophic typo.
4. Use a Terminal Multiplexer: Use tmux or screen on a bastion host. The last thing you need is for a dropped SSH connection to leave you with a half-applied, broken configuration.
5. Verify, Change, Verify: Run your “show” commands first to capture the “before” state (e.g., show ip bgp summary, show ip route). Execute your change. Immediately run the same commands again to confirm the “after” state is what you expected.

Warning: This method is still stressful and prone to human error. It’s a necessary stopgap, not a long-term strategy. It reduces risk, but it doesn’t eliminate it. It doesn’t scale and it feeds the culture of fear.

Solution 2: The ‘Infrastructure as Code’ Sanity Saver

This is where we actually fix the problem. Treat your network configuration exactly like you treat your application code. Your router config is just a text file, after all. It belongs in Git.

The workflow should look familiar:

1. Git Repo: All router and switch configurations are stored in a Git repository. The `main` branch is your source of truth.
2. Feature Branch: To make a change, you create a new branch. E.g., `feature/add-customer-xyz-subnet`.
3. Pull Request (PR): You make your config change on the branch and open a PR. This is where the magic happens. Your team can review the exact change (`diff`), comment on it, and suggest improvements. No more cowboy changes.
4. Automated Tooling: When the PR is approved and merged, a CI/CD pipeline kicks in. A tool like Ansible (with its network modules), Nornir, or even Terraform connects to the device and applies the change idempotently.

Here’s a taste of what an Ansible task to manage a prefix-list might look like. Instead of manually typing commands, you define the desired state.

- name: Configure IOS prefix-list for customer-xyz
  cisco.ios.ios_config:
    lines:
      - ip prefix-list CUSTOMER_TRANSIT_OUT seq 10 permit 198.51.100.0/24
      - ip prefix-list CUSTOMER_TRANSIT_OUT seq 20 permit 203.0.113.0/25
    parents: []
    match: line
    replace: block

This approach transforms the process. Changes are small, peer-reviewed, audited, and applied consistently by a machine. You’ve made the change process boring, which is exactly what you want.

Pro Tip: For the truly advanced, look into tools like Batfish. You can integrate it into your CI pipeline to simulate the network-wide impact of your configuration change *before* it ever touches a production router. It’s like having a unit test for your network.

Solution 3: The ‘It’s Not My Problem Anymore’ Cloud Pivot

Sometimes, the best solution is to change the game entirely. I want you to ask yourself a hard question: “Does my company derive a competitive advantage from managing BGP sessions by hand?” For 99% of us, the answer is a resounding “No.”

This is the cloud-native approach. Instead of managing the complex, low-level routing protocols yourself, you consume networking as a managed service.

This is a strategic shift. It’s about deciding to focus your engineering effort on your product, not on being a miniature ISP. It’s not for everyone, but if you’re already heavily invested in the cloud, you should be aggressively trying to offload this kind of undifferentiated heavy lifting to your cloud provider.

So, how often should you make changes to L3 routing protocols? As often as the business needs you to. Our job isn’t to be gatekeepers of a fragile system. Our job is to build a resilient, automated system that makes change the default, not the exception.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions