Creating a Python Lambda Layer As a Zip File

A Step-by-Step Guide

AWS Lambda Layers — Image credits to google

When working with AWS Lambda functions, managing dependencies can sometimes feel like navigating a complex maze. Lambda layers provide an elegant solution to this challenge, allowing you to package and manage libraries separately from your function code. In this guide, I’ll walk you through the process of creating a Python Lambda layer, ensuring your Lambda functions have all the necessary dependencies without cluttering your core function code.

Prerequisites

Before we begin, make sure you have:

• Python 3.10 installed
• pip package manager
• AWS CLI configured (optional, but recommended)
• Basic familiarity with Python virtual environments

Step 1: Create a Version-Specific Virtual Environment

Start by creating a dedicated virtual environment for your project. This ensures a clean, isolated workspace for installing dependencies:

Important: Ensure your virtual environment uses the exact same Python version as your Lambda function and layer. For example, if you’re creating a Lambda layer for Python 3.10, you must:

• Use Python 3.10 to create the virtual environment
• Create the layer with Python 3.10 runtime
• Ensure your Lambda function also uses Python 3.10 runtime

python3.10 -m venv .venv

Step 2: Activate the Virtual Environment

Activate your newly created virtual environment:

source .venv/bin/activate

Step 3: Install and Record Dependencies

For this example, we’ll install xlrd and fastparquet. Install the packages and then freeze your requirements:

pip install xlrd fastparquet
pip freeze > requirements.txt

This command creates a requirements.txt file that captures the exact versions of your installed packages.

Step 4: Prepare Lambda Layer Directory

Create the nested directory structure required for a Lambda layer:

mkdir -p lambda_layer/python/lib/python3.10/site-packages

The specific nested structure is crucial for Lambda to recognize and properly load your libraries.

Step 5: Install Dependencies in Layer Directory

Install your requirements directly into the Lambda layer directory:

pip install -r requirements.txt -t lambda_layer/python/lib/python3.10/site-packages

This command installs all listed packages into the layer’s site-packages directory.

Step 6: Compress the Lambda Layer

Create a ZIP archive of your layer:

cd lambda_layer
zip -r lambda_layer.zip .

Step 7: Upload the Layer in AWS Console

Navigate through the AWS Lambda console:

1. Go to the Layers section
2. Click Create layer
3. Choose a descriptive name
4. Upload the ZIP file you just created
5. Select Python 3.10 as the runtime
6. Click Create

Step 8: Add Layer to Lambda Function

Finally, attach the layer to your Lambda function:

1. Go to Functions
2. Select your target Lambda function
3. Scroll to the Layers section
4. Click Add a layer
5. Choose Custom layers
6. Select your newly created layer
7. Pick the appropriate version
8. Click Add

Pro Tips

• Always match your layer’s Python runtime (version) with your Lambda function’s runtime.
• Mismatched versions can cause runtime errors
• Use python --version to confirm your Python version
• Keep your layers lean and focused
• Version your layers to manage updates and rollbacks
• Use layers for common libraries across multiple functions

Conclusion

Lambda layers simplify dependency management, making your serverless applications more modular and easier to maintain. By following these steps, you can efficiently package and deploy complex Python libraries to your AWS Lambda functions.

Happy serverless coding! 🚀