Struggling with a web application that has zero API documentation? A senior engineer details three battle-tested methods—from browser dev tools to reverse proxy logging—to reverse-engineer and document any black-box API you encounter.

The Black Box API: A DevOps Guide to Documenting Undocumented Web Apps

It was 2 AM. A critical payment processing feature was failing in production, and the only person who knew how it worked had left the company six months prior. We had the frontend code, but the backend API it called was a complete mystery. No docs, no OpenAPI spec, nothing. That night, hunched over a browser’s network tab, I learned a lesson that’s saved my bacon more times than I can count: every API can be reverse-engineered. You just need to know where to look.

So, How Did We Get Here?

Let’s be real, this situation is painfully common. It’s the ghost of tech debt past. Maybe you’ve inherited a legacy system from a team that’s long gone. Maybe your company acquired another’s tech stack and the “documentation” was a single README file. Or maybe the original team just never got around to it. The root cause doesn’t matter when you’re the one on call. The result is the same: a functional web application that’s a ‘black box’ from an API perspective. You can see what it does, but you have no idea how it communicates with its backend services.

Fear not. We’re going to crack this box open. I’ve got three reliable methods in my toolkit, ranging from a quick peek to a full-scale surveillance operation.

Solution 1: The Archaeologist’s Toolkit (Browser Dev Tools)

This is your first and fastest line of attack. It’s built into every modern browser, and it’s surprisingly powerful. You’re essentially watching the conversation between the frontend and backend as it happens.

1. Open the Inspector: Right-click anywhere on the web app and hit “Inspect,” then navigate to the “Network” tab.
2. Filter the Noise: You’ll see a flood of requests for images, CSS, etc. Click the “Fetch/XHR” filter. This narrows it down to just the API calls we care about.
3. Perform an Action: Click a button in the UI, like “Save Profile” or “Load Dashboard.”
4. Examine the Evidence: You’ll see new entries pop up in the network log. Click on one. You can now inspect everything:
   • Headers: Check the Request URL (e.g., /api/v2/users/12345/profile), the Request Method (GET, POST, PUT?), and authorization tokens (e.g., Authorization: Bearer ...).
   • Payload/Body: For POST or PUT requests, this tab shows you the exact JSON structure the frontend is sending. This is gold.
   • Response: This tab shows you what the server sent back. The structure of this data is just as important as the request.

Pro Tip: Once you find a request you want to replicate, right-click on it in the Network tab and select “Copy as cURL”. You can paste this directly into your terminal to replay the request, or import it into a tool like Postman to start building out your own documentation collection. It’s a massive time-saver.

This method is fantastic for quick, targeted investigations. Its main limitation is that it only shows you what you do. To see everything, we need to go deeper.

Solution 2: The Wiretap (Reverse Proxy Logging)

If the browser is a snapshot, this is the 24/7 security camera. The idea is to place a reverse proxy, like Nginx or Caddy, between the users and the actual application server (e.g., app-server-03). All traffic has to pass through our proxy, and we can configure it to log everything we need.

This is my preferred method for building a comprehensive picture. Here’s a simplified Nginx example. First, we define a custom log format in our nginx.conf that captures the request body:

http {
    # ... other http settings ...

    log_format api_log_detailed escape=json
      '{'
        '"timestamp":"$time_iso8601",'
        '"client_ip":"$remote_addr",'
        '"request_method":"$request_method",'
        '"request_uri":"$request_uri",'
        '"status":$status,'
        '"body_bytes_sent":$body_bytes_sent,'
        '"http_referer":"$http_referer",'
        '"http_user_agent":"$http_user_agent",'
        '"request_body":"$request_body"'
      '}';

    # ...
}

Then, in our server block, we tell Nginx to proxy requests to the real application and use our new log format.

server {
    listen 443 ssl;
    # ... ssl certs and other configs ...
    
    server_name my-legacy-app.techresolve.com;

    # This is crucial for reading the request body
    client_max_body_size 10M;

    location / {
        # The actual app server we are proxying to
        proxy_pass http://app-server-03:8080;
        
        # Log all requests using our detailed format
        access_log /var/log/nginx/api.log api_log_detailed;
    }
}

Now, every single API call that hits our application gets logged to /var/log/nginx/api.log with the full request URI and body. We can use tools like tail, grep, or a log aggregator to analyze traffic patterns over time and discover endpoints we never would have found manually.

Solution 3: The Interceptor (Man-in-the-Middle Proxy)

Sometimes, the traffic doesn’t come from a browser. It might be a thick-client desktop app or a mobile application. In these cases, you can’t use browser dev tools and setting up a reverse proxy for an entire mobile fleet isn’t feasible. This is where you bring out the heavy artillery: a man-in-the-middle (MitM) proxy tool like mitmproxy, Charles, or Fiddler.

These tools run on your local machine and configure your OS (or mobile device) to route all network traffic through them. They can intercept, inspect, and even modify any HTTP/S request on the fly. This gives you the same level of detail as the browser’s Network tab, but for any application on your device.

Warning: This is the “nuclear” option for a reason. You are actively intercepting and decrypting encrypted traffic. This requires installing a custom root certificate on your device, which has major security implications. NEVER do this on a machine that handles sensitive personal data, and only ever use it in a controlled, isolated development environment. You have been warned.

Which Method Should You Choose?

It depends on your needs. I’ve put together a quick cheat sheet:

Documenting the undocumented isn’t glamorous work, but it’s a critical skill for any engineer working with real-world systems. It transforms a legacy black box from a source of fear into a system you can understand, maintain, and improve. Start with the browser, scale up to a proxy when you need more data, and you’ll be the hero who finally brought the map to the treasure hunt.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions