Exporting WhatsApp Chat Logs to Structured Data (JSON)

Hey there, Darian Vance here. For a while, our on-call support handoffs were tracked in a dedicated WhatsApp group. It was functional, but a nightmare to audit. I was spending at least an hour every Monday morning scrolling, copy-pasting, and manually building reports on issue volume and response times. It was a huge time sink.

I realized I could automate this whole process. I built a simple Python utility to parse those exported chat logs and turn them into clean, structured JSON. Now, the process is fully automated and feeds directly into our internal analytics dashboard. It’s saved my team countless hours. Today, I’m going to walk you through how you can build the same thing. This is a high-value, low-effort task that pays dividends fast.

Prerequisites

• Python 3.6 or newer installed on your machine.
• An exported WhatsApp chat log file (a .txt file). You can get this by going into a chat, tapping the three-dot menu, selecting ‘More’, then ‘Export chat’, and choosing ‘Without Media’.
• A basic understanding of Python. Familiarity with regular expressions is a plus, but I’ll break it down for you.

The Guide: Step-by-Step

Step 1: Project Setup

First things first, get your workspace ready. I’ll skip the standard virtualenv setup since you likely have your own workflow for that. Just make sure you’re working in an isolated environment. The only third-party library we’ll need for this is Pandas, which is an absolute powerhouse for data manipulation. You can add it to your project with a simple pip command.

Once your environment is active, create a new Python file. I’ll call mine parser.py.

Step 2: The Core Logic – Parsing the Raw Text

If you open the exported .txt file, you’ll see that each message generally follows a pattern: [DATE, TIME] SENDER: MESSAGE. Our goal is to capture each of these components. The main challenge? This format can be inconsistent, and messages can span multiple lines. We’ll use a regular expression (regex) to reliably break each line down.

Here’s the full script. Let’s place this inside your parser.py file.

import re
import pandas as pd
import json

def parse_whatsapp_chat(file_path):
    # This regex is the key. It captures the date, time, sender, and message.
    # It's designed to handle both 12-hour and 24-hour formats to some extent.
    # Group 1: Datetime, Group 2: Sender, Group 3: Message
    line_regex = re.compile(r'^(\d{1,2}/\d{1,2}/\d{2,4}, \d{1,2}:\d{2}(?:\s?[apAP][mM])?) - ([^:]+): (.*)$')
    
    parsed_data = []
    with open(file_path, 'r', encoding='utf-8') as f:
        for line in f:
            match = line_regex.match(line)
            if match:
                # A new message line was found
                parsed_data.append({
                    'timestamp': match.group(1).strip(),
                    'sender': match.group(2).strip(),
                    'message': match.group(3).strip()
                })
            elif parsed_data:
                # This is a continuation of the previous message (multi-line)
                # We append it to the 'message' of the last recorded entry.
                parsed_data[-1]['message'] += '\n' + line.strip()
    
    return parsed_data

def main():
    # Replace '_chat.txt' with the actual name of your exported file
    chat_file = '_chat.txt'
    
    try:
        raw_data = parse_whatsapp_chat(chat_file)
    except FileNotFoundError:
        print(f"Error: The file '{chat_file}' was not found. Please make sure it's in the same directory.")
        return # Replaces sys.exit()

    if not raw_data:
        print("No data was parsed. Check the file format and content.")
        return

    # Convert the list of dictionaries to a Pandas DataFrame
    df = pd.DataFrame(raw_data)
    
    # --- Data Cleaning (Optional but Recommended) ---
    # Convert timestamp string to a proper datetime object
    df['timestamp'] = pd.to_datetime(df['timestamp'], format='%d/%m/%Y, %I:%M %p', errors='coerce')
    
    # Filter out any rows where timestamp conversion failed
    df.dropna(subset=['timestamp'], inplace=True)
    
    print(f"Successfully parsed {len(df)} messages.")

    # Export the DataFrame to a JSON file
    # orient='records' creates a list of JSON objects, which is very useful.
    df.to_json('whatsapp_chat.json', orient='records', indent=4, date_format='iso')
    
    print("Exported data to whatsapp_chat.json")

if __name__ == "__main__":
    main()

Pro Tip: Understanding the Regex

The regex `^(\d{1,2}/\d{1,2}/\d{2,4}, \d{1,2}:\d{2}(?:\s?[apAP][mM])?) – ([^:]+): (.*)$` might look intimidating, but it’s simple when you break it down:

• `^`: Asserts the start of the line.
• `(\d{1,2}/\d{1,2}/\d{2,4}, \d{1,2}:\d{2}(?:\s?[apAP][mM])?)`: This is Group 1, capturing the full timestamp. It looks for date formats like DD/MM/YYYY and time formats like HH:MM AM/PM. The `(?:…)` is a non-capturing group for the optional AM/PM part.
• ` – `: Matches the literal separator.
• `([^:]+)`: This is Group 2, the sender. It captures any character that is not a colon.
• `: `: Matches the colon and space after the sender’s name.
• `(.*)$`: This is Group 3, the message. It captures everything else until the end of the line.

Step 3: Running the Script

Save your exported WhatsApp file as _chat.txt in the same directory as your parser.py script. Then, simply run the script from your terminal:

python3 parser.py

If all goes well, you’ll see a success message and a new file named whatsapp_chat.json will appear. This file contains your cleanly structured chat data, ready for any analysis you can dream of.

Common Pitfalls

Here are a couple of things that tripped me up when I first built this, so you can avoid them.

• Timestamp Format Hell: WhatsApp’s timestamp format can vary based on your phone’s language and region settings (e.g., DD/MM/YY vs. MM/DD/YY, or 24-hour time). If the script fails, the first place to check is the regex and the `format` string in `pd.to_datetime`. You may need to tweak it to match your specific export file.
• System Messages: The script currently doesn’t differentiate between user messages and system messages like “John Doe joined the group” or “Messages and calls are end-to-end encrypted.” These lines don’t match the regex and are ignored. In my production setups, I often add logic to specifically filter or flag these system messages after the initial parse if I need to account for them.
• Character Encoding: Most exports are UTF-8, but sometimes they aren’t. If you get a `UnicodeDecodeError`, it means the file has a different encoding. You can try opening the file with `errors=’ignore’` in the `open()` function, but be aware this might discard some special characters or emojis.

Pro Tip: Automation with Cron

In my production environment, I don’t run this manually. I have a process that automatically places the exported file in a specific directory, and a cron job runs the parser script nightly. A simple cron entry looks like this, running every Monday at 2 AM:

0 2 * * 1 python3 parser.py

This keeps our data fresh without any manual intervention.

Conclusion

And that’s it. You’ve turned a messy, unstructured text file into a clean, machine-readable JSON array. From here, the possibilities are endless. You can load this data into a database, feed it into a visualization tool like Grafana or Tableau, or perform complex analysis to track keywords, user activity, and response times.

This is a perfect example of a small DevOps automation that delivers significant value by saving manual labor and unlocking new insights from existing data. I hope this helps you reclaim some of your time.

Happy scripting,
Darian Vance

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions