Struggling with Shopify webhook timeouts for video processing? This guide breaks down why it happens and provides three actionable solutions, from a quick-and-dirty fix to a fully scalable, event-driven serverless architecture using SQS and Lambda.

Shopify Webhooks, Video Processing, and the 3 AM Pager Duty Alert

I still remember the night clearly. It was 2:47 AM and the on-call phone, buzzing angrily on my nightstand, jolted me awake. A flood of PagerDuty alerts: `504 Gateway Timeout` on our primary webhook endpoint. A quick check of the logs confirmed my suspicion. The marketing team, bless their hearts, had just launched a massive campaign for a new product line, bulk-uploading hundreds of product videos directly in Shopify. Our monolithic PHP app, which naively tried to download, transcode, and push each video to TikTok’s API right when the webhook hit, was completely overwhelmed. Each request took 30-60 seconds, while Shopify gives you maybe 5. We were drowning in retries. It was a self-inflicted DDoS attack, and a painful lesson I’ll never forget.

The Root of the Problem: You’re Making Shopify Wait

I see this all the time. A junior dev gets a task: “When a product video is added in Shopify, post it to Reels.” They write a single script at an endpoint, `api/v1/hooks/product-updated`. The script does everything: receives the webhook, downloads the video, runs it through FFMPEG, uploads it to a new service, and then finally returns a `200 OK`. The problem is, you’re holding the connection open with Shopify the entire time.

Shopify, like any good webhook provider, isn’t going to wait around forever. It expects you to acknowledge receipt of the event almost instantly. When you don’t, it assumes your service is down and does two things, both of which are bad for you:

• It marks the delivery as failed.
• It schedules a retry, adding even more load to your already-struggling server.

Your job isn’t to do the work when the webhook arrives. Your job is to accept the work and say “Thanks, got it!” as fast as humanly possible.

Solution 1: The “Stop the Bleeding” Fix (Acknowledge & Defer Locally)

It’s 3 AM, the system is on fire, and you just need it to work. This is the battlefield fix. It’s not pretty, but it will get you back to sleep. The idea is to have your endpoint do the absolute bare minimum: grab the request body and save it somewhere to be processed later. Then, immediately send a `200 OK` back to Shopify.

In this scenario, we’ll just dump the JSON payload into a database table called `webhook_jobs`.

The Webhook Endpoint Code (e.g., in Node.js/Express)

// endpoint: /api/v1/hooks/product-updated
app.post('/product-updated', async (req, res) => {
  // 1. VERY basic validation (in reality, verify the HMAC signature!)
  const videoUrl = req.body.video?.src;
  if (!videoUrl) {
    return res.status(400).send('Bad Request: No video source found.');
  }

  // 2. Immediately insert the job into a database table
  try {
    await db.query(
      'INSERT INTO webhook_jobs (payload, status) VALUES ($1, $2)',
      [req.body, 'pending']
    );
    // 3. IMPORTANT: Acknowledge success IMMEDIATELY
    res.status(202).send('Accepted'); 
  } catch (error) {
    // If the DB write fails, we have a real problem.
    console.error('Failed to queue webhook job:', error);
    res.status(500).send('Internal Server Error');
  }
});

Then you’d have a separate cron job that runs every minute, queries this `webhook_jobs` table for ‘pending’ jobs, and processes them one by one.

Is this hacky? Yes. It makes your web server stateful and doesn’t scale well. If your single `worker.sh` script gets backed up, the queue just grows. But it separates the acknowledgement from the processing, which is the critical first step.

Solution 2: The “Right Way” (Decouple with a Message Queue)

This is the solution I’d expect any mid-level or senior engineer to propose. Instead of a database table on the same server, we use a dedicated message queue service like AWS SQS (Simple Queue Service) or RabbitMQ. This properly decouples your webhook ingestion from your video processing.

The architecture looks like this:

1. Shopify sends a webhook to your API endpoint.
2. Your API endpoint receives the payload, validates it, and immediately pushes it as a message to an SQS queue. This takes milliseconds.
3. Your API endpoint returns a `200 OK` to Shopify.
4. A completely separate group of servers (your “workers”) are constantly polling this SQS queue for new messages.
5. When a worker gets a message, it performs the long-running video processing task.

This is resilient. If your workers are down, the messages just pile up safely in the queue until the workers come back online. You can scale the number of workers up or down based on the queue depth, without ever touching your webhook ingestion API.

The Webhook Endpoint Code (Using AWS SDK)

// Using AWS SDK v3 for Node.js
import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";

const sqsClient = new SQSClient({ region: "us-east-1" });
const QUEUE_URL = "https://sqs.us-east-1.amazonaws.com/123456789012/MyVideoProcessingQueue";

app.post('/product-updated', async (req, res) => {
  // Always verify the Shopify HMAC signature in production!
  const messageBody = JSON.stringify(req.body);

  const command = new SendMessageCommand({
    QueueUrl: QUEUE_URL,
    MessageBody: messageBody,
    MessageGroupId: req.body.id // Useful for FIFO queues
  });

  try {
    await sqsClient.send(command);
    res.status(202).send('Accepted');
  } catch (error) {
    console.error("Failed to send message to SQS:", error);
    res.status(500).send('Internal Server Error');
  }
});

Pro Tip: Idempotency is Key. Shopify *will* occasionally send duplicate webhooks. Design your worker logic so that processing the same event twice doesn’t cause a duplicate TikTok post. Use the product ID or video ID as a key to check if you’ve already processed this job.

Solution 3: The “Serverless Revolution” (Event-Driven Architecture)

This is where we go from fixing a problem to building a truly modern, scalable, and cost-effective system. We get rid of the dedicated server running our API endpoint entirely. This is the “cloud-native” approach.

The flow becomes even more abstract and powerful:

1. Shopify sends a webhook to an Amazon API Gateway endpoint.
2. API Gateway is configured to trigger an AWS Lambda function.
3. The Lambda function’s only job is to validate and push the event to SQS. It runs for maybe 50ms and then shuts down. You only pay for those 50ms.
4. The SQS queue then triggers a second Lambda function (or an AWS Fargate container if your video processing takes longer than 15 minutes).
5. This second “processor” function does the heavy lifting: downloads from Shopify’s URL, transcodes, and uploads to TikTok/Reels.

This architecture is the gold standard for handling event-based workloads. It scales automatically from zero to thousands of requests per second without you managing a single server. The cost is directly proportional to the number of webhooks you receive.

Comparison of Solutions

Ultimately, the right choice depends on your scale, your team’s expertise, and your budget. But please, for the sake of your on-call engineer, stop doing heavy lifting directly inside your webhook endpoints. Acknowledge and defer. Your future self will thank you.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions