Datadog’s Cloud Cost Management is powerful but reactive. To truly control cloud spend, you need to shift from monitoring costs to proactively enforcing budget policies and tag hygiene through Infrastructure as Code and automated remediation.

Datadog Cost Management Saved My Bacon… And Then I Realized It Wasn’t Enough

I still remember the Monday morning meeting. The VP of Engineering slid a printout across the table, a Datadog graph with a cost spike that looked like an EKG during a heart attack. “Darian,” he said, “What is ‘prod-data-migration-test-cluster-04’ and why did it cost us five grand this weekend?” Turns out, a junior engineer had spun up a massive EKS cluster for a “quick test” on Friday, picked the wrong instance types, and went home for the weekend. Datadog’s Cost Management tool caught it, sure. It sent an alert. But by the time we saw it, the money was already spent. The tool told us *that* we were bleeding, but it didn’t apply the tourniquet. That’s when I realized that just *observing* cost is a losing game.

The ‘Why’: Your Cost Tool is a Rear-View Mirror

Listen, tools like Datadog Cloud Cost Management are fantastic for visibility. They slice and dice your bill better than the cloud providers’ native tools and help you attribute costs. But at their core, they are reporting tools. They tell you what happened yesterday. The root of the problem isn’t the tool; it’s the process. The core issues are almost always:

• Lack of Tagging Discipline: Without consistent and mandatory tags (e.g., team, project, environment), all the beautiful dashboards in the world are just showing you a giant, unattributable pile of money you’ve spent. You can’t fix what you can’t identify.
• Reactive vs. Proactive Stance: Getting an alert that you’ve already overspent by 50% is useful for the next budget meeting, but it doesn’t get your money back. True cost control happens *before* the resource is even created or when it first violates a policy.

Taming the Beast: 3 Ways to Actually Control Costs

So, you’re getting the alerts but the bill is still climbing. Let’s get our hands dirty. Here are the three levels of intervention we use at TechResolve, from the quick fix to the permanent solution.

Fix #1: The Quick & Dirty – Hyper-Specific Alerts

The first step is to move beyond the default “Total spend is over $X” alerts. They’re too broad. You need to get granular and create alerts that signify *intent* and *anomaly* before they become a disaster. Instead of one big alert, create dozens of small ones.

Think about setting up monitors in Datadog for things like:

• Untagged High-Cost Resources: “Alert when any EC2 instance larger than m5.2xlarge is launched without a ‘project’ tag.”
• Anomalous Service Spend: “Alert when daily spend for S3 bucket ‘dev-asset-uploads’ exceeds its weekly average by 200%.”
• Zombie Resource Detection: “Alert when a group of instances tagged with ‘env:temp-test’ have been running for more than 24 hours.”

Pro Tip: Be careful not to create alert fatigue. Route these specific alerts directly to the team responsible. A cost alert for the ‘data-science’ project should go to their Slack channel, not the general ops channel where it will be ignored.

Fix #2: The Grown-Up Solution – Enforce Tagging with IaC

Alerts are good, but preventing the problem is better. This is where you stop asking nicely and start enforcing rules through code. If you’re using Terraform, this is non-negotiable. Use your provider configuration to mandate tags on *every single resource* you manage.

Here’s a snippet from our AWS provider configuration. If a developer tries to `terraform apply` a resource without these three tags, the plan fails. Period. The resource is never created.

provider "aws" {
  region = "us-east-1"

  default_tags {
    tags = {
      ManagedBy   = "Terraform"
      Environment = "Unknown"
      Project     = "Unknown"
      Team        = "Unknown"
    }
  }
}

# In a separate tfvars or variable definition, you can enforce specific values
# But the simplest way is to make them mandatory in the module itself.
#
# Example of a module enforcing a specific tag:
variable "team_name" {
  type        = string
  description = "The name of the team responsible for this resource."
  validation {
    condition     = length(var.team_name) > 2
    error_message = "A team_name tag must be provided."
  }
}

This simple change shifts the responsibility left. It’s no longer Ops’ job to chase down untagged resources; it’s the developer’s job to declare ownership before they can even deploy.

Fix #3: The ‘Nuclear’ Option – Automated Remediation

Okay, let’s talk about the option that makes managers nervous but engineers cheer. For non-production environments, sometimes the only way to teach is to, well, break things. We use cloud-native tools like AWS Budgets Actions or a Lambda function triggered by a CloudWatch event to automatically take action.

The logic is simple: IF a new EC2 instance is launched in the `dev` account AND it does not have a `owner` tag AND it has been running for more than 60 minutes, THEN execute the `ec2:TerminateInstances` API call on it.

Here’s a conceptual comparison:

WARNING: I cannot stress this enough. DO NOT point an automated termination script at your production environment unless you have a 1000% confidence level in your tagging and logic. This is for cleaning up dev sandboxes and preventing weekend-long billing accidents, not for managing `prod-db-01`.

It’s a Culture, Not Just a Tool

At the end of the day, Datadog’s Cost Management tool is an essential part of our stack. It gives us the visibility we need to ask the right questions. But it’s not a silver bullet. True cloud cost control isn’t about buying another tool; it’s about building a culture of ownership. It’s about making the cost of a resource as visible to a developer as a failing unit test. Use your tools to enforce that culture, not just to report on the bill afterward.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions