Deploy custom Python package on AWS Lambda

So what I am trying to do is to call the open weather API every 6 hours and store the data in a MongoDB database. This article is mainly focused on how to create the python package.

Note: If your lambda is configured with VPC you need to make sure to follow the post and this post to configure it correctly.

There are a few pre-requisites

  1. Make sure Gitbash is installed if using windows. It gives Unix style command-line environment.
  2. Make sure 7zip is installed and added to the PATH. By this, if you type 7z and hit enter you see —

Now let’s come to the packaging part, make sure you are using a virtual environment. You can easily create one using (make sure using Gitbash if windows)

$ python -m venv venv
$ source venv/Scripts/activate

So when you will see a prompt, you can find (venv) written before the prompt like this —

Suppose the folder structure is like that

temperature_monitor/
|--venv/
|--lambda_function.py

My lambda_function.py is a simple python script that basically fetches data from openweathermap.org and stores it inside MongoDB.

when we do pip list we can see the 3rd party packages installed in the virtual environment.

Now you can run your script and it executes successfully.

Now comes the packaging part. I have seen most of the people just putting the lambda_function.py inside Lib/site-packages and making the zip, but there are many other packages inside the site-packages folder that we don’t need. here is an example of the zip sizes —

As you can see, the site-packages_large.zip is created using the method mentioned just now and it is much larger (5MB compared to 2MB of the method that I will discuss now).

I have created one script to automate this, given the folder structure —

temperature_monitor/
|--venv/
|--lambda_function.py

create a file called package_maker.sh and paste the code inside it

# just checking if the filename is not null
if [ -z "$1" ]
then
echo "You did not put the file name"
echo "it should be like ./package_maker.sh <file-name>"
exit 1
fi
# creating the requirements.txt file
pip freeze > requirements.txt
# removing any pre existing lambda_package folder
rm -rf ./lambda_package
# making new lambda_package folder
mkdir ./lambda_package
# copy the main python file into the directory
cp $1 ./lambda_package/$1
# installing the packages into that directory
pip install --target=./lambda_package -r requirements.txt
# creating the zip
echo "make sure 7z.exe is added in your path"
echo "it normally resides in C:\Program Files\7-Zip"
7z a -r -tzip lambda_package.zip ./lambda_package/*# removing the lambda_package directory
rm -rf ./lambda_package
# completed
echo "completed, find the zip named lambda_package.zip"

It is very simple and self-explanatory. This script will basically create a temporary folder named lambda_package/ and put only the specific packages and create the zip named lambda_package.zip and delete the temporary folder. To use this script type ./package_maker.sh lambda_function.py inside in your git bash. ./ before package_maker.sh is important, because without this it will search inside PATHs.

(venv)probhakarsarkar@xxxx~/Desktop/works/temperature_monitor $ ./package_maker.sh lambda_function.py.
.
.
.
Everything is Ok
completed, find the zip named lambda_package.zip

You can change the function name from lambda_function.py and the callable inside it from lambda_handler to anything else but make sure to change it accordingly inside runtime settings —

Now that you have created the lambda_package.zip t is just a matter of uploading it into lambda —

After the lambda is deployed and tested successfully we can configure the CloudWatch event to schedule it —

Then just configure details > create rule . It will call the lambda every 6 hours.

--

--

--

Love to tinker with things. https://www.probhakarsarkar.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Your Guide to Regular Expressions in Python and Terminal

The Magic Of Matrix

PILLAR OF COURSERA :AWS

Better Portals Mod for Minecraft 1.15.2/1.12.2 — Wminecraft.net

Better Portals Mod for Minecraft 1.15.2/1.12.2 for Minecraft

How to Set up a VPN on iOS Device?

Turn Swagger Theme to the Dark Mode

Building a high-performing Engineering team

From Alexa To a call center with Amazon Connect, Lex and Jovo

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Probhakar

Probhakar

Love to tinker with things. https://www.probhakarsarkar.com

More from Medium

AWS Lambda with Python for CRUD Operation

The Fastest way to load Files into AWS S3 Buckets: A Guide to Using Amazon Command Line Interface

Mock AWS DynamoDB service for unit testing in Python

Migrating a WSGI Python application to Serverless on AWS