The Locust Kubernetes Operator v2.0 is a game-changer with its Go rewrite, offering blistering speed and OpenTelemetry support. Here’s how we navigated the zero-downtime migration and why it’s worth your time.

The Locust K8s Operator v2.0 Upgrade: From Sluggish Python to Blazing Go

I still get a nervous twitch thinking about it. It was 2 AM, the final go-live deployment for ‘Project Chimera’ was on the line, and our entire performance testing pipeline ground to a halt. The old Python-based Locust operator, bless its heart, was taking nearly ten minutes to reconcile and spin up the worker pods for our complex, multi-host scenario. Ten minutes where the entire release team was staring at a blinking cursor in Slack, waiting for my all-clear. We made it, but barely. That night, I swore I’d find a better way. When I saw the v2.0 Go rewrite hit, I knew that moment had arrived.

So, What Was The Real Problem?

Let’s be clear, the original Locust operator was a fantastic tool that got the job done. But it was built in Python with Kubebuilder. For most things, that’s fine. But in a Kubernetes controller, you’re in a constant state of reconciliation—watching for changes and acting on them. The Python Global Interpreter Lock (GIL) and the general overhead of an interpreted language meant that as we scaled up our number of LocustTest custom resources, the controller just got… sluggish. It wasn’t broken, it was just hitting the limits of its architecture. The rewrite in Go solves this at a fundamental level. Go is compiled, statically typed, and built for concurrency with goroutines. This means the new operator is a lean, mean, pod-creating machine that reacts almost instantly.

Navigating the Upgrade: Three Paths Forward

Migrating a core piece of your testing infrastructure can feel like open-heart surgery. Thankfully, the team behind the operator made this incredibly smooth. Here are the three ways we approached it across different environments, from dev to our most critical production-adjacent clusters.

Solution 1: The ‘By The Book’ Zero-Downtime Migration

This is the officially recommended path, and I can confirm it works exactly as advertised. The beauty is that the new v2 controller is designed to adopt the existing v1 LocustTest custom resources seamlessly. It’s the least disruptive and should be your default choice.

The Steps:

1. First, and I can’t stress this enough, back up your existing LocustTest resources. A quick `kubectl get locusttests –all-namespaces -o yaml > locust-backup.yaml` will save your bacon if things go sideways.
2. Apply the new v2.0 Custom Resource Definition (CRD). This updates the schema in Kubernetes to support the new features, but it’s backward compatible.

   kubectl apply -f https://raw.githubusercontent.com/LocustLoadTesting/locust-operator/v2.0.0/config/crd/bases/locust.io_locusttests.yaml

3. Deploy the new v2.0 Controller. This is the magic step. The new Go-based controller will start up, see the existing CRs, and take over management from the old Python controller without missing a beat.

   kubectl apply -f https://raw.githubusercontent.com/LocustLoadTesting/locust-operator/v2.0.0/config/operator/operator.yaml

4. Once you’ve confirmed everything is stable, you can safely uninstall the old v1 operator deployment.

Pro Tip: Watch the logs of the new operator pod right after you apply it (`kubectl logs -f -n locust-operator-system -l control-plane=controller-manager`). You should see it immediately start reconciling your existing tests. It’s oddly satisfying.

Solution 2: The ‘Clean Slate’ Re-deploy

Sometimes you just don’t trust an in-place upgrade. Maybe you’ve got some weird state from old experiments, or you just prefer the peace of mind that comes from a fresh start. This path is more disruptive, as it involves a brief downtime for your tests, but it guarantees you’re starting from a known-good state.

The Steps:

1. Scale down all your active Locust tests to 0 workers to stop any active runs.

   kubectl scale locusttest my-checkout-test --replicas=0

2. Back up all your LocustTest YAML definitions (you did this in the first solution, right?).
3. Completely uninstall the v1 operator and its CRD.
4. Install the shiny new v2 operator and its CRD from scratch, as per their documentation.
5. Re-apply your saved LocustTest YAML files. The new controller will pick them up and create them fresh.

We used this approach in our development cluster (`dev-perf-k8s-01`) where we had a lot of cruft lying around. It felt good to clean house.

Solution 3: The ‘Trust but Verify’ Canary Rollout

For our most critical, pre-production environment, we couldn’t afford any surprises. So, we ran both operators side-by-side in a canary-style deployment. This is the most complex approach, but it’s also the safest for zero-risk tolerance environments.

The trick is to isolate the new v2 operator so it doesn’t fight with the v1 operator over the same resources.

The Steps:

1. Create a new namespace for validation, let’s call it `locust-v2-canary`.
2. Download the `operator.yaml` for v2.0 locally instead of applying it directly from the URL.
3. Modify the downloaded `operator.yaml`. Find the Deployment resource for the `locust-operator-controller-manager` and add a `–namespace` argument to the container command, pointing it to your canary namespace.

   # In your modified operator.yaml
   ...
   spec:
     containers:
     - command:
       - /manager
       args:
       - --leader-elect
       - --namespace=locust-v2-canary # <-- ADD THIS LINE
   ...

4. Apply your modified manifest. Now, the v2 operator is running, but it will only watch for and manage LocustTest resources inside the `locust-v2-canary` namespace. The v1 operator continues managing everything else.
5. Deploy a few representative tests into the `locust-v2-canary` namespace and validate everything. Check the startup times, look at the OpenTelemetry data, and get comfortable.
6. Once you’re confident, you can proceed with the full rollout using Solution 1 and then decommission your canary setup.

This is my personal favorite for anything that touches production. It lets you see the benefits firsthand in a controlled slice of your environment before committing completely. The performance difference was immediately obvious—our test startup time went from minutes to under 30 seconds. That’s a win you can take to the bank.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions