Tired of manual CLI changes being overwritten by your deployment pipeline? This guide explains why the “YAML vs. CLI” debate matters and provides three battle-tested strategies to manage your cloud infrastructure declaratively and avoid configuration drift for good.

YAML vs. The Command Line: A Senior Engineer’s Guide to Winning the War on Configuration Drift

I still remember the pager going off at 3:17 AM. A junior engineer, let’s call him Mark, had been trying to fix a performance issue on our `prod-user-service`. He saw the pods were CPU-throttled, so he did what seemed logical: he ran a quick kubectl scale deployment prod-user-service --replicas=8. The fire was out. He went to bed a hero. An hour later, our CI/CD pipeline ran its scheduled sync, looked at the `user-service.yaml` file in our Git repo which still said `replicas: 3`, and “fixed” the cluster back to its “correct” state. The service fell over again, the pager screamed, and I spent the next hour on a call explaining the concept of “source of truth” to a very tired, very confused Mark. This isn’t a rare story; it’s a rite of passage in the world of modern infrastructure, and it’s born from the fundamental conflict between working imperatively versus declaratively.

The Real Problem: Imperative Commands vs. A Declarative World

This whole “CLI vs. YAML” thing isn’t just about preference. It’s a clash of two different philosophies.

• The Command Line (Imperative): You are giving a direct order. “Hey Kubernetes, scale this deployment to 8 pods. Do it now.” The cluster obeys, no questions asked. The problem is, it has no memory of why you did it, and this state exists only in the live cluster.
• YAML Files (Declarative): You are describing a desired state. “Hey Kubernetes, the state of this deployment should be 8 replicas.” You check this file into Git. It becomes the permanent, agreed-upon source of truth.

Configuration drift happens when the live, imperative state (what you did with the CLI) no longer matches the declarative source of truth (what’s in your Git repo). When your automation (like a CI/CD pipeline) runs, it will always trust Git over the live state, ruthlessly reverting your manual changes. It’s not a bug; it’s the system working exactly as designed to enforce consistency.

Solution 1: The Quick & Dirty Fix (The “Backport”)

Sometimes, you’re in a firefight. The site is down, and you need to make a change right now. You don’t have time to go through a full pull request process. I get it. We’ve all been there. This is the emergency glass you break, but you have to clean up the mess immediately after.

The Workflow:

1. Make the live change: Use the imperative command to put out the fire. The `edit` command is often better than `scale` or `patch` because it opens the full object for you.

   kubectl edit deployment prod-api-gateway

2. Immediately export the new state: Once the change is applied and verified, you need to pull that new configuration out of the cluster and get it into a file.

   kubectl get deployment prod-api-gateway -n production -o yaml > prod-api-gateway.yaml

3. Commit the truth: Now, clean up that exported YAML (remove the system-generated fields like `status`, `resourceVersion`, etc.) and commit it back to your Git repository. This makes your “emergency” change the new official source of truth.

Darian’s Warning: This is a reactive, break-glass-in-case-of-emergency procedure. If this becomes your team’s standard operating procedure, you don’t have a process; you have a recurring accident. It’s a useful skill for an emergency, but a terrible habit.

Solution 2: The Right Way (The “YAML-First” Discipline)

This is the grown-up solution and the one we enforce at TechResolve. It requires discipline from the team, but it eliminates 99% of configuration drift issues. The rule is simple: The cluster is read-only for humans. You never, ever run a `kubectl` command that changes state directly on a production environment. All changes are proposed via YAML in a Git repository.

Here’s how it compares:

This process gives you peer review, an audit trail (Git history), and a single source of truth. Your live cluster state is now a direct reflection of what’s in your `main` branch. No more surprises.

Solution 3: The ‘Gold Standard’ (The GitOps Gauntlet)

This is the final evolution. You take the “YAML-First” approach and put it on steroids with a dedicated GitOps tool like Argo CD or Flux. These tools install an operator in your cluster whose only job is to watch your Git repository and mercilessly enforce its state on the cluster.

How it works:

• You point your GitOps controller at your Git repo (e.g., `github.com/TechResolve/infra-manifests`).
• The controller has read-only access to the repo and admin rights in the Kubernetes cluster.
• It constantly compares the live state of the cluster with the desired state defined in the YAML files in Git.
• If it detects any difference (a.k.a. “drift”), it automatically corrects the cluster.

Imagine Mark from my story earlier. In a GitOps world, he could have run his `kubectl scale` command, and within 30 seconds, Argo CD would have detected the change, seen that Git still said `replicas: 3`, and automatically scaled the deployment back down. It makes manual CLI changes completely futile.

Pro Tip: Setting up GitOps is the ultimate way to enforce discipline. It changes the conversation from “Please don’t use the CLI to change things” to “You can’t use the CLI to make a lasting change, so don’t even bother.” It forces everyone to adopt the correct YAML-first workflow.

Look, the command line is an invaluable tool for debugging—for running `kubectl get logs`, `describe pod`, or `exec`-ing into a container. But for changing state, it’s a liability. Embrace the declarative nature of your YAML files, lock them down in Git, and let automation do the rest. Your sleep schedule will thank you.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions