Struggling with Amazon ECS tasks that start before their dependencies are truly ready? I’ll break down the common `dependsOn` trap and give you three real-world solutions to fix it for good.

The ECS `dependsOn` Trap: Why Your Service Fails Even When You Did Everything Right

I’ll never forget the 3 AM PagerDuty alert. A brand new service, `order-processor`, was in a complete crash loop. The logs showed it couldn’t connect to the database. “Impossible,” I thought, “I have a perfect `dependsOn` condition in the task definition. The `order-processor` can’t start until the `db-migrator` task finishes successfully.” I checked the ECS console. Sure enough, `db-migrator` was stopped, reason: “Essential container exited.” Exit code: 0. Success. So why was everything on fire? That night, I learned a hard lesson: ECS’s definition of “success” and our definition were two very different things.

The Root of the Problem: The “Essential Container” Lie

When you set up a dependency in your ECS Task Definition using `dependsOn`, you’re telling the ECS scheduler, “Don’t start Task B until Task A meets a specific condition.” The most common conditions are `SUCCESS` or `HEALTHY`. Here’s the trap: ECS only cares about the status of the one container marked essential: true in Task A’s definition.

Let’s imagine your Task A is a database initialization task. It has two containers:

• An `init-script` container (marked `essential: true`) that runs a quick schema update and exits with code 0.
• A `cloud-sql-proxy` sidecar container that actually maintains the connection to your database.

When you start this task, the `init-script` runs in two seconds and exits cleanly. ECS sees this, says “Great! The essential container Succeeded!”, and immediately starts your main application, Task B. The problem is, the `cloud-sql-proxy` sidecar might still be starting up or establishing its connection tunnel. Your application wakes up, tries to connect, and fails because the proxy isn’t ready. Cue the crash loop.

Pro Tip from the Trenches: A task’s `dependsOn` condition is met the moment its essential container either passes its health check (`HEALTHY`) or stops with an exit code of 0 (`SUCCESS`). ECS does not wait for non-essential sidecars to become healthy or stop.

Three Ways to Fix This Mess

After you’ve stopped swearing at your monitor, you have a few ways to solve this for good. They range from a quick-and-dirty fix to a proper architectural change.

Solution 1: The “Pray and Wait” (The Quick Hack)

This is the simplest, ugliest, but sometimes necessary fix to stop the bleeding during an outage. You add a `sleep` command to the entrypoint of your dependent container (your main application).

Instead of your Dockerfile’s entrypoint being `[“/app/start-server”]`, you’d change it to `[“/app/entrypoint.sh”]` and create that script:

#!/bin/sh
# entrypoint.sh for the main application container

echo "Waiting 15 seconds for dependencies to settle..."
sleep 15

echo "Starting application."
exec /app/start-server

Why it’s a hack: You’re just guessing. Maybe 15 seconds is enough today, but during a high-load deployment, the proxy might take 20 seconds. It’s a brittle, non-deterministic fix that should make you feel a little dirty. But hey, sometimes you just need to get the system stable at 3:30 AM.

Solution 2: The Resilient Application (The “Right” Way)

The most robust, cloud-native solution is to make your application itself responsible for handling unavailable dependencies. It shouldn’t just crash if the database isn’t there on the first try. It should retry with a backoff strategy.

Your application’s startup logic should look something like this (pseudocode):

max_retries = 5
wait_time = 2 // seconds

for i in 1..max_retries:
    try:
        db_connection = connect_to_database(connection_string)
        log("Successfully connected to the database!")
        break // Exit the loop on success
    except ConnectionError as e:
        log("Failed to connect to DB, attempt %d/%d. Retrying in %d seconds...", i, max_retries, wait_time)
        sleep(wait_time)
        wait_time = wait_time * 2 // Exponential backoff

if not db_connection:
    log("Could not establish DB connection after %d retries. Shutting down.", max_retries)
    exit(1)

// Proceed with starting the web server...
start_server(db_connection)

Why it’s better: This makes your application resilient to transient failures, not just slow startups. If `prod-db-01` has a momentary blip, your app will recover gracefully instead of entering a crash loop and triggering alarms. The logic for handling dependencies lives where it belongs: in the service that has the dependency.

Solution 3: The “Truthful” Dependency (The Orchestration Fix)

This approach fixes the problem at the source. You make the essential container in your dependency task (Task A) tell the truth. It shouldn’t exit until all its components, including sidecars, are *actually ready*.

You accomplish this with a wrapper script as the entrypoint for your essential container. This script starts the primary process in the background and then enters a loop, checking the health of its sidecars or downstream connections before finally exiting.

Here’s an example for a task that depends on a local proxy sidecar being available on port `5432`:

#!/bin/sh
# wrapper-script.sh for the essential container in the dependency task

# Start the main process for this container in the background
/usr/bin/run-migrations &

# Wait for the main process to finish
wait $!

echo "Migrations complete. Now checking for proxy sidecar health..."

# Use netcat (nc) to poll the sidecar port
while ! nc -z localhost 5432; do
  echo "Waiting for proxy sidecar on port 5432..."
  sleep 2
done

echo "Proxy sidecar is ready. Exiting successfully."
exit 0

Why it’s powerful: This makes ECS `dependsOn` behave exactly as you’d expect. The essential container’s lifecycle now accurately reflects the readiness of the *entire task*. It’s more complex to set up but keeps the retry logic out of your primary application code, which can be a good separation of concerns.

Which Should You Choose?

Here’s a quick breakdown to help you decide:

My advice? Always strive for Solution 2. Building resilient, self-healing applications is a core tenet of modern cloud architecture. Use Solution 3 for cases where you can’t modify the application code (e.g., third-party software) and only use Solution 1 to put out a fire before you implement a real fix.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions