Struggling to test the new Kubernetes v1.35 RC1? I break down three real-world strategies, from a quick local setup to a full-blown GitOps approach, to help you validate new features without blowing up production.

So, You Want to Test Kubernetes v1.35? A Senior Engineer’s Guide to Not Breaking Everything

I still remember the “Great Staging Silence of ’22”. We were upgrading from K8s 1.23 to 1.24. We read the release notes, skimmed the “deprecations” section, and figured it was a routine bump. We ran our basic smoke tests—pods came up, services were reachable. Green lights everywhere. Then, an hour later, our monitoring dashboards for the staging environment lit up like a Christmas tree. A subtle change in how `NetworkPolicy` handled a specific `ipBlock` rule had silently black-holed all traffic between our core application and its database `prod-db-clone-01`. It took two engineers half a day to trace it. All because our “testing” was just a glorified health check. That’s why seeing a Reddit thread about people jumping into v1.35 RC1 makes me want to share a few hard-won lessons.

Why “Just Reading the Docs” Is a Recipe for Disaster

Look, the Kubernetes release team does a phenomenal job with documentation. But a changelog can’t capture the infinite complexity of your specific environment. The root cause of most upgrade failures isn’t a glaring bug in the new version; it’s a subtle interaction between a change in Kubernetes and your unique combination of CNIs, CSIs, controllers, and application workloads. A feature flag that’s benign in a vanilla cluster might conflict with the specific way your Istio sidecars are configured. You’re not just testing Kubernetes; you’re testing your Kubernetes.

Solution 1: The Sandbox Spin-up (Your Quick & Dirty Local Lab)

This is the first-pass, “does this even work?” sanity check. Forget your massive cloud-based clusters for a moment. We’re talking about spinning up a multi-node Kubernetes cluster right on your laptop in under two minutes. My tool of choice here is `kind` (Kubernetes in Docker) because it’s fast and disposable.

This approach is perfect for testing a very specific new feature, like a new Pod scheduling directive or an alpha API field. It’s isolated, it’s fast, and if you break it, you just run kind delete cluster and start over.

# kind-config-v1.35.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  # Point to the specific RC1 image
  image: kindest/node:v1.35.0-rc.1 
- role: worker
  image: kindest/node:v1.35.0-rc.1
- role: worker
  image: kindest/node:v1.35.0-rc.1

Then, just create it:

kind create cluster --config kind-config-v1.35.yaml --name k8s-rc1-test

Heads Up: This is a hacky but effective way to get your hands on the new version. It’s not a real-world test. Your local Docker networking is not your cloud provider’s VPC, and a `kind` cluster doesn’t have the IAM roles, storage classes, and other integrations that your real clusters depend on.

Solution 2: The Dedicated Proving Ground (The Staging Cluster Upgrade)

This is the grown-up version of testing. You have a dedicated staging cluster, let’s call it k8s-staging-west2, that’s a smaller but architecturally similar version of production. It runs the same observability stack, the same CNI, the same ingress controllers. Here, the goal is to test the upgrade process itself and then validate your actual applications against the new version.

Our process at TechResolve looks like this:

1. Backup Everything: We take a full cluster backup using Velero. Snapshots are non-negotiable.
2. Control Plane First: We upgrade the control plane nodes one by one, watching the API server logs like a hawk for unusual errors.
3. Node Pool Canary: We have a small, non-critical ‘canary’ node pool. We upgrade just those nodes first.
4. Deploy Test Apps: We deploy a suite of our core applications to that canary pool and run a full battery of integration and performance tests. We’re not just checking if pods run; we’re checking if our billing ingestion pipeline can still process 10,000 messages per minute.
5. Full Rollout: Only after the canary pool is stable for a day do we proceed with upgrading the remaining worker nodes.

Pro Tip: Before you even start the upgrade, deploy a tool like Pluto to your staging cluster. It will scan for deprecated Kubernetes API versions in your deployed Helm charts and manifests, catching things that the upgrade pre-flight checks might miss.

Solution 3: The Ephemeral Cluster-as-Code Approach (The ‘Nuke and Pave’ Method)

This is the ‘nuclear option’, but it’s also the gold standard for repeatable, automated testing. Instead of upgrading a long-lived staging cluster, you define your entire cluster infrastructure as code using tools like Terraform or Cluster API (CAPI). The testing process becomes part of your CI/CD pipeline.

A pipeline job can:

1. Provision a brand new VPC and all the networking.
2. Use Terraform or CAPI to build a fresh Kubernetes v1.35.0-rc.1 cluster from scratch.
3. Deploy your entire application stack using your GitOps tooling (ArgoCD, Flux).
4. Run a comprehensive end-to-end test suite.
5. Report the results and, crucially, tear the entire thing down.

This is powerful because it eliminates configuration drift and ensures you’re testing against a pristine environment every single time. It’s how you can say with 100% confidence that your application stack is compatible with the new K8s version, from the ground up.

Which Path Should You Choose?

Honestly, it depends on your team’s maturity and the criticality of the cluster. Here’s my breakdown:

No matter what, don’t be the engineer who brings down staging because a changelog “looked fine”. Spin up a test environment, get your hands dirty with the release candidate, and test what actually matters: your own applications. Your future self will thank you.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions