Amazon Lambda for those not familiar with it is defined as an event driven serverless platform for compute resources. Rather than paying for an EC2 instance or running a docker instances on ECS that runs executable code, Lambda offers an alternative that is for many much less expensive. For instance if you need to routinely resize photos that are provided to you from various distributed sources, you can create an S3 bucket for images to be loaded to, which can act as a trigger for a Lambda job to resize them and output the resized photos into a different bucket.
There are a plethora of useful things you can leverage Lambda for. You can use it as a backend service for mobile applications that need to call databases, post data, etc. Lambda can also be used on a cron to run an EBS snapshot backup routinely as well as snapshotting RDS and replicating snapshots to another region to facilitate a disaster recovery scenario. Lambda can also be built to run repeated jobs such as resizing images, reacting to cloudwatch metrics and actioning accordingly (rebooting systems with pegged CPU, etc), or hitting an analytics API and loading the parsed data into a data warehouse.
I’m going to cover supported languages, some of the ways Lambda can be invoked, as well as a discussion of different framework options available to incorporate Lambda into your Infrastructure as code strategy, setting up a cron based Lambda to snapshot EBS volumes, and finally some gotchas of using Lambda.
Supported Languages
First and foremost the languages supported in AWS Lambda at the time of this writing include the following Node.js, Java, C# and Python(2.x and 3.x). To keep up to speed with supported languages for Lambda you can refer to their FAQ page located here.
Lambda Invocation Methods
Methods for invoking Lambas currently includes CloudWatch alerts, Cron style invocation, S3 trigger, API Gateway triggers, SES and others. A full list of AWS services that can act as Lambda triggers can be found here.
Lambda/Serverless Frameworks
Over the past few years a number of frameworks have emerged for building and maintaining serverless resources in AWS. I will break these down into broadly 2 catagories, the first category is a more traditional approach to infrastructure as code in which you define each of the peices needed for your lambda operation and use the framework to generate CloudFormation templates and apply them. The second category is frameworks specifically built to update and maintain your Lambda and serverless resources such as API Gateway and Dynamo DBs throughout their full life cycle.
Infrastructure as Code Frameworks:
Serverless Frameworks:
Your mileage may vary with each of these choices. Tools like Claudia, Node Lambda, and Serverless are built on node and will require npm and node locally to run. Tools such as Zappa and Chalice are Python based. Each of these frameworks supports more than their native language in Lambda, however depending on your workflow, development environment, and any CI processes you have this may sway your decision.
Lambda Gotchas
As with any cloud service offering there are configuration maximums and limitations to be aware of. It is best to be aware of these limitaitons and maximums ahead of time so there are no suprises down the road as demand increases for your application and your scaling needs change. The information outlined here is based off of current limitations, please ensure you are using the most up to date information by checking here.
Service Limits:
- Invocations Per Second: 100 (this limit can be increased by requesting a limit increase from AWS support, please note this can take more than a day)
- Concurrent Executions: 1000 per region
- Max Execution Duration: 300 seconds
- Memory Allocation: Min:128mb Max:1536MB
- Deploy Package Size(compressed): 50MB
- Total Size of All Deployment Packages Per Region:75GB
Some other things worth mentioning are that if you are running a cron based Lambda function for things such as snapshotting EBS volumes it is important to monitor these jobs, as the size of the backup set grows it will take longer to execute the lambda function. You can setup monitoring on an invididualized bassis in CloudWatch. While these individualized monitors are a fantastic idea it is also worth setting up some generalized blanket monitoring such as lambda invocation rate per minute and Lambda error rate. This way you are alerted when you are approaching your service limites for Lambda invocations and can request service increases, or be aware if there is a spike in Lambda error rates.
Setting up EBS Snapshotting Using Lambda and Terraform
Rather than endlessley rambling about all the cool features and things you can do with AWS I wanted to create an actual get your hands dirty dive into Lambda. For this lab we’ll setup a Lambda running on cron trigger to snapshot our EBS volumes and another lambda to delete snapshots that are more than 90 days old. For this example we will set this up using terraform. To follow along visit the Github project here, follow the instructions in the readme and profit! The code for the snapshot and snapshot management is originally from this repo.
Final Thoughts
Hopefully this post as has been helpful in seeing ways you can use Lambda and what tools and frameworks are available to use with your Lambda projects. Thank you for taking the time to read this post, I look forward to creating future blogs around serverless resources and other infrastructure as code topics.