Building a side project that relies on a third-party API like Harvest is a great way to learn, but it also exposes the harsh realities of external integrations. This guide breaks down how to handle unreliable APIs, moving from quick fixes to robust, production-ready solutions.

That Time a Partner API Paged Me at 3 AM

I’ll never forget it. 2:47 AM on a Tuesday. PagerDuty screaming on my nightstand. The alert? “CRITICAL: Customer data sync failed for 15 consecutive minutes.” I roll out of bed, log in, and see our primary data sync worker, `sync-worker-prod-01`, just hammering a partner’s API and getting nothing but `503 Service Unavailable` back. Our logs were a wall of red. The worst part? It wasn’t our fault. The partner was doing “unscheduled maintenance.” My quick, “bulletproof” cron job had no concept of failure beyond a simple retry, so it just kept slamming the door, waiting for someone to answer.

That’s the exact trap I see developers fall into, especially on side projects like the “weekly planner for freelancers” I saw on Reddit. You build this awesome tool, it works great on your machine, but you forget one fundamental truth: you don’t control the third-party API. It will fail. It will be slow. It will change without telling you. Your application’s resilience is only as strong as its weakest, flakiest integration.

So, What’s the Real Problem Here?

The core issue is coupling and optimism. You’ve coupled your application’s core logic directly to a network call you can’t guarantee will succeed. When a user clicks “Sync with Harvest,” your code makes a direct, synchronous request. If that request hangs or fails, the user is stuck staring at a spinner, and your backend is tied up waiting for a response that might never come.

You’re being optimistic, assuming the network is reliable and the third-party service is always available. In production, that optimism will get you paged at 3 AM. We need to plan for failure, not just hope for success.

Fixing It: From Duct Tape to Reinforcement

Let’s walk through how to handle this, from a quick patch to a solution that will actually let you sleep through the night.

1. The Quick Fix: “Client-Side Retry with Hope”

This is the most basic, “get it working for the demo” approach. You wrap your API call in a function that retries a few times if it fails. It’s better than nothing, but it still locks up the user’s browser and has no visibility on the backend.

Let’s say you have a simple `fetch` call in your JavaScript:

// The "optimistic" way
async function syncWithHarvest(data) {
  const response = await fetch('/api/harvest-sync', {
    method: 'POST',
    body: JSON.stringify(data),
    headers: { 'Content-Type': 'application/json' }
  });
  if (!response.ok) {
    throw new Error('Sync failed!');
  }
  return response.json();
}

The quick fix is to add a simple retry loop. This is a hack, but it’s a common one.

// The "slightly less optimistic" way
async function syncWithHarvestWithRetry(data, retries = 3, delay = 1000) {
  for (let i = 0; i < retries; i++) {
    try {
      const response = await fetch('/api/harvest-sync', { /* ... */ });
      if (response.ok) {
        return await response.json();
      }
      // Only retry on server errors or network issues
      if (response.status < 500) {
        throw new Error(`Client Error: ${response.status}`);
      }
    } catch (error) {
      console.error(`Attempt ${i + 1} failed. Retrying in ${delay}ms...`);
      if (i === retries - 1) throw error; // Rethrow last error
      await new Promise(res => setTimeout(res, delay * (i + 1))); // Exponential backoff
    }
  }
}

When to use this: Honestly? Almost never in production. Maybe for a non-critical GET request, but for anything that modifies data (a POST or PUT), this is just asking for trouble.

2. The Permanent Fix: “Decouple with a Job Queue”

This is how we do it in the real world. The user’s action should not trigger the API call directly. Instead, it should add a “job” to a queue. A separate “worker” process then picks up that job and handles the integration, complete with robust retries, logging, and error handling.

The flow looks like this:

1. User clicks “Sync.”
2. Your frontend sends the request to your server (e.g., `/api/harvest-sync`).
3. Your server does NOT call the Harvest API. Instead, it creates a job with the necessary payload and pushes it onto a queue (like RabbitMQ, AWS SQS, or even a Redis list).
4. Your server immediately responds to the user with “OK, we’ve scheduled your sync!” The UI can now update instantly.
5. A separate worker process, running on a server like `worker-jobs-prod-01`, pulls the job from the queue.
6. The worker attempts the API call to Harvest. If it fails (e.g., with a `429 Too Many Requests` or `503 Service Unavailable`), the queueing system can automatically reschedule the job to run again later, often with exponential backoff.

Pro Tip: This pattern is called Asynchronous Task Processing. It’s the backbone of almost every scalable web application. It makes your app feel faster to the user and infinitely more resilient to external failures.

Here’s a conceptual Node.js/Express example using a hypothetical queue library:

// In your Express route handler (api/harvest-sync.js)
import { harvestSyncQueue } from '../queues';

app.post('/api/harvest-sync', async (req, res) => {
  const userId = req.user.id;
  const dataToSync = req.body;

  // Add a job to the queue instead of calling the API directly
  await harvestSyncQueue.add('sync-user-data', { userId, dataToSync });
  
  // Respond to the user immediately
  res.status(202).json({ message: 'Sync has been scheduled!' });
});


// In your worker process (worker.js)
import { harvestSyncQueue } from './queues';
import { callHarvestApi } from './harvest-client';

harvestSyncQueue.process('sync-user-data', async (job) => {
  const { userId, dataToSync } = job.data;
  console.log(`Processing sync for user ${userId}`);
  
  // This function contains the actual API call logic, with retries.
  // The queue will handle retrying the entire job if this throws an error.
  await callHarvestApi(dataToSync); 
});

3. The ‘Nuclear’ Option: “The Anti-Corruption Layer”

Sometimes, an API isn’t just flaky, it’s truly awful. It has a bizarre data structure, inconsistent error formats, or requires three separate calls to do one logical thing. In these extreme, business-critical cases, we build a dedicated service to act as a buffer between our application and the chaotic third-party API.

This is known as an Anti-Corruption Layer or an Adapter pattern. It’s a microservice whose only job is to communicate with that one specific, painful API.

This is a heavy-duty solution. You’re building and deploying an entirely new service. But when an integration is both critical to your business and pathologically unreliable, this is the only way to protect your core application and maintain your sanity.

Warning: Don’t reach for this first. It’s complex. A well-implemented job queue (Option 2) solves 95% of these problems. Only build an adapter when the API’s design itself, not just its availability, is causing you constant grief.

So next time you’re integrating an external service, stop and think. Don’t be the person who gets paged at 3 AM because you were too optimistic. Plan for failure, decouple your systems, and you’ll build something that lasts.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions