Detecting Large File Uploads in S3 and Alerting via Slack

Hey team, Darian here. A few quarters back, I was spending way too much time sifting through CloudTrail logs to track down rogue uploads. A client’s automated backup script went haywire and uploaded a 500GB file to a standard S3 bucket, and we didn’t notice until the bill came. That was the last straw. I built this simple serverless workflow to get instant Slack alerts for large files, and it’s saved me countless hours and a few headaches since. It’s a classic “set it and forget it” solution that provides immediate value.

Let’s walk through how you can set this up in about 15 minutes.

Prerequisites

• An AWS account with permissions for S3, Lambda, and IAM.
• A Slack workspace where you have permission to add apps.
• Python 3.8+ and the AWS CLI installed on your local machine for testing and deployment, if you prefer that over the console.

The Step-by-Step Guide

Step 1: Create a Slack Incoming Webhook

First, we need a way to send messages to a Slack channel. The easiest and most secure way is with an Incoming Webhook.

1. Navigate to api.slack.com/apps and click “Create New App”. Choose “From scratch”.
2. Give your app a name (e.g., “S3 Large File Notifier”) and select your workspace.
3. In the app’s settings, go to “Incoming Webhooks” under the “Features” section and activate it.
4. Click “Add New Webhook to Workspace”, choose a channel for the notifications, and authorize it.
5. Slack will generate a Webhook URL. It will look something like https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX.

Crucially, treat this URL as a secret. We will store it securely in our Lambda function’s configuration, not in the code itself.

Step 2: Create the IAM Role for Lambda

Our Lambda function needs permission to run and write logs to CloudWatch. That’s it. S3 will *push* events to Lambda, so the function doesn’t need explicit `s3:GetObject` permissions just to receive the notification, which is a common point of confusion. The event payload itself contains all the metadata we need, including the file size.

1. In the AWS Console, go to IAM > Roles > “Create role”.
2. For the trusted entity type, select “AWS service”. For the use case, choose “Lambda”.
3. On the “Add permissions” page, search for and add the AWSLambdaBasicExecutionRole policy. This gives our function permission to write logs.
4. Give the role a descriptive name, like S3LargeFileNotifierRole, and create it.

Step 3: Write the Lambda Function Code

Now for the core logic. I’ll skip the standard virtualenv setup since you likely have your own workflow for that. The key libraries you’ll need are `boto3` (which is included in the Lambda runtime) and `urllib3` for sending the web request. You don’t need to package `urllib3` either, as it’s also available in the standard runtime.

Here’s the Python code. The logic is simple:

1. Parse the incoming S3 event to get the bucket name, object key, and size.
2. Check if the size exceeds a threshold we define.
3. If it does, format a message and send it to our Slack webhook URL.

import os
import json
import urllib.parse
import urllib3

# Initialize the HTTP client
http = urllib3.PoolManager()

# Get environment variables
SLACK_WEBHOOK_URL = os.environ['SLACK_WEBHOOK_URL']
FILE_SIZE_LIMIT_MB = int(os.environ['FILE_SIZE_LIMIT_MB'])

# Convert MB to Bytes for comparison
SIZE_LIMIT_BYTES = FILE_SIZE_LIMIT_MB * 1024 * 1024

def lambda_handler(event, context):
    try:
        # Extract bucket and object key from the S3 event record
        record = event['Records'][0]
        bucket_name = record['s3']['bucket']['name']
        
        # The object key can have special characters, so we unquote it
        object_key = urllib.parse.unquote_plus(record['s3']['object']['key'], encoding='utf-8')
        
        # The file size is included directly in the event!
        object_size = record['s3']['object']['size']

        print(f"Processing file: {object_key} from bucket: {bucket_name}. Size: {object_size} bytes.")

        if object_size > SIZE_LIMIT_BYTES:
            print(f"File exceeds size limit of {FILE_SIZE_LIMIT_MB} MB. Sending Slack alert.")
            
            # Convert size to a more readable format for the message
            size_in_mb = round(object_size / (1024 * 1024), 2)
            
            # Create a direct console link to the object
            region = record['awsRegion']
            console_link = f"https://s3.console.aws.amazon.com/s3/object/{bucket_name}?region={region}&prefix={object_key}"

            message = {
                "text": f":warning: Large file detected in S3 bucket `{bucket_name}`",
                "blocks": [
                    {
                        "type": "section",
                        "text": {
                            "type": "mrkdwn",
                            "text": f":warning: *Large File Upload Detected!*"
                        }
                    },
                    {
                        "type": "section",
                        "fields": [
                            {"type": "mrkdwn", "text": f"*Bucket:*\n`{bucket_name}`"},
                            {"type": "mrkdwn", "text": f"*File Size:*\n{size_in_mb} MB"},
                            {"type": "mrkdwn", "text": f"*File Name:*\n`{object_key}`"}
                        ]
                    },
                    {
                        "type": "actions",
                        "elements": [
                            {
                                "type": "button",
                                "text": {
                                    "type": "plain_text",
                                    "text": "View S3 Object"
                                },
                                "url": console_link,
                                "style": "primary"
                            }
                        ]
                    }
                ]
            }
            
            encoded_message = json.dumps(message).encode('utf-8')
            headers = {'Content-Type': 'application/json'}
            
            resp = http.request('POST', SLACK_WEBHOOK_URL, body=encoded_message, headers=headers)
            
            print(f"Slack API response: {resp.status}")

        else:
            print("File size is within the acceptable limit. No action taken.")

        return {
            'statusCode': 200,
            'body': json.dumps('Processing complete.')
        }

    except Exception as e:
        print(f"An error occurred: {e}")
        # To prevent retries for a malformed event, we return a success status code
        # but log the error for our own debugging.
        return {
            'statusCode': 200, 
            'body': json.dumps(f'Error processing event: {str(e)}')
        }

Pro Tip: Notice how we’re reading `SLACK_WEBHOOK_URL` and `FILE_SIZE_LIMIT_MB` from `os.environ`. This is a best practice. It lets us change these values without ever touching the code. For production setups, I strongly recommend storing the webhook URL in AWS Secrets Manager and having the Lambda function fetch it at runtime for maximum security.

Step 4: Deploy the Lambda Function & Configure a Trigger

1. In the AWS Console, navigate to Lambda and click “Create function”.
2. Choose “Author from scratch”.
3. Name your function (e.g., `s3-large-file-alerter`).
4. Select “Python 3.9” (or a newer version) as the runtime.
5. Under “Permissions”, choose “Use an existing role” and select the S3LargeFileNotifierRole you created in Step 2.
6. Once the function is created, paste the Python code into the `lambda_function.py` code editor. Click “Deploy” to save your changes.
7. Go to the “Configuration” tab, then “Environment variables”. Add two variables:
   • Key: SLACK_WEBHOOK_URL, Value: (Your Slack webhook URL from Step 1)
   • Key: FILE_SIZE_LIMIT_MB, Value: 100 (or whatever threshold you want)
8. Finally, let’s add the trigger. In the “Function overview” section, click “+ Add trigger”. Select “S3” as the source.
9. Choose the bucket you want to monitor.
10. For “Event types”, select “All object create events”. Acknowledge the recursive invocation warning and click “Add”.

That’s it! Now, whenever a new file is uploaded to your bucket, this Lambda will fire, check its size, and alert you on Slack if it’s over the limit.

Where I Usually Mess Up (Common Pitfalls)

• IAM Permissions: Forgetting to attach the execution role is the #1 mistake. If you don’t see logs in CloudWatch, it’s almost always a permissions issue.
• S3 Event Triggers: Make sure you are triggering on object creation events (`PUT`, `POST`, `COPY`, `MultipartUpload`). If you trigger on `GET` requests, you’ll invoke the function constantly and burn through your free tier.
• Environment Variables: A typo in the environment variable name in your code or in the Lambda configuration will cause a `KeyError`. Double-check that `SLACK_WEBHOOK_URL` matches exactly.
• Object Key Encoding: File names can contain spaces or special characters (`my test file.jpg`). The S3 event key is URL-encoded. Forgetting to use `urllib.parse.unquote_plus` will lead to incorrect file names in your Slack alerts.

Conclusion

You’ve just built a robust, serverless monitoring system. This simple pattern of S3 → Lambda is incredibly powerful for all sorts of automation, not just alerting. You can use it to kick off image processing, data ingestion, or archival workflows. It’s a fundamental tool in my DevOps toolkit, and now it’s in yours too.

Happy automating!

– Darian

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions