Lambda on Technology Enlightened

Mirror HTTP request using ALB backed by AWS Lambda

Sun, 12 Sep 2021 10:30:00 +0000

When Gonzalo Diethelm asked me whether there is a way to echo the http request to validate Lambda@Edge functions that modifies the request by adding / updating / deleting headers? I couldn’t find something that’s already available. Hence I quickly created a simple Lambda function front-ended by Application Load Balancer to just return the response that it has recieved.

Serverless UI testing using Selenium, AWS Lambda, AWS Fargate, and AWS Developer Tools

Sat, 20 Mar 2021 08:30:00 +0000

Since the post Using AWS CodePipeline, AWS CodeBuild, and AWS Lambda for Serverless Automated UI Testing was published, things have evolved with Chrome headless and Firefox headless being supported natively. AWS Lambda now supports container images, AWS Step Functions has added support for Map state and its integration with Lambda, and AWS Fargate has enabled automating the UI testing completely using serverless technologies.

The goal of this post is to demonstrate how to use AWS Developer Tools) to build a continuous delivery pipeline that automatically deploys a test environment and runs a UI test against it. We also explain how to build a container image which includes Chromium browser, Firefox browser, Selenium, and its dependencies, which are used in Lambda and Fargate. You can accelerate the testing by launching multiple test cases in parallel using the Map state in Step Functions.

Using AWS CodePipeline, AWS CodeBuild, and AWS Lambda for Serverless Automated UI Testing

Sat, 23 Sep 2017 08:21:28 +0000

Testing the user interface of a web application is an important part of the development lifecycle. In this post, I’ll explain how to automate UI testing using serverless technologies, including AWS CodePipeline, AWS CodeBuild, and AWS Lambda.

I built a website for UI testing that is hosted in S3. I used Selenium to perform cross-browser UI testing on Chrome, Firefox, and PhantomJS, a headless WebKit browser with Ghost Driver, an implementation of the WebDriver Wire Protocol. I used Python to create test cases for ChromeDriver, FirefoxDriver, or PhatomJSDriver based the browser against which the test is being executed.

Resources referred to in this post, including the AWS CloudFormation template, test and status websites hosted in S3, AWS CodeBuild build specification files, AWS Lambda function, and the Python script that performs the test are available in the serverless-automated-ui-testing GitHub repository.

Monitor AWS Trusted Advisor Checks

Tue, 14 Mar 2017 17:39:24 +0000

AWS Trusted Advisor is an important tool which provides more than 50 checks across four different categories Cost Optimization, Performance, Security & Fault Tolerance (at the moment of writing this post).

But I have noticed that this is not being used more extensively considering the importance of it. Hence I created couple of lambda functions which shall be configured to run on daily basis to refresh the checks and create summary of all the checks and send an email.

Scheduling automated EBS snapshots serverless using AWS Lambda

Thu, 09 Jun 2016 11:46:37 +0000

EBS snapshot as backup mechanism is a very common practice and it also has been defined in Backup and Recovery section of EC2 best practices. If you have read my another article Save AWS costs by scheduled start and stop of EC2 instances, similar approach will be used in creating Snapshots too.

Using JSON in EC2 Tags enables us to provide granular configuration details. Create an EC2 tag with name as backup and value with details containing time to take backup and retention period. Obtaining these details from Tags will enable us to configure different schedule & retention period for different EC2 instance. If no backup is needed on a particular day, that day shall be removed from the value.

Sync S3 bucket to multiple S3 buckets in different region

Fri, 27 May 2016 11:55:41 +0000

Cross-Region Replication for Amazon S3 was introduced last year which enables replicating objects from a S3 bucket to a different S3 bucket located in different region (it can be same/different AWS account). It’s one of the most sought after feature for long time but it doesn’t solve some of the scenarios as defined below.

Synchronizing to a bucket in same region.
Synchronizing to multiple destination buckets.
Object ACLs needs to be set at the source bucket hence multiple different ACLs needs to be set when copying across different accounts.
Replication doesn’t change the ownership of the objects hence bucket policy applied at the destination bucket won’t be honored.

Refer the feedback from AWS support team regarding the last point mentioned above.

Using AWS IoT button - Simple Example

Sat, 14 May 2016 17:32:27 +0000

Amazon did a limited release of the programmable Dash button and unsurprisingly it got sold out within a day. It’s stated in amazon.com that “We don’t know when or if this item will be back in stock.”, hence you might need to wait to get one.

I take this opportunity to write a post about how I used the AWS IoT button that I received in AWS re:Invent 2015 to show case AWS IoT to the team.

Automated Blue/Green deployment using Lambda and CloudFormation

Thu, 05 May 2016 07:42:54 +0000

Blue/Green deployment is a well-known method to deploy an application without any downtime. Performing DNS switch is one of the very common techniques to achieve this. Using DNS switch has a minor issue with DNS caching which might take some time for DNS change to be propagated.

Apart from DNS switch, AWS gives us two different options to switch the stacks. One is to have single ELB and swap it across auto-scaling group and another is switching the launch configuration of the auto-scaling group. Check the following presentation on various methods to achieve blue/green deployment in AWS.

Sending response back to CFN custom resource from python Lambda function

Sun, 20 Dec 2015 06:00:56 +0000

CloudFormation uses a pre-signed S3 URL to receive the response back from the custom resources managed by it. There are few blue prints available for Node.js Lambda custom resources but nothing available for python yet. Hence I created this simple function which shall be used to send the response back to CFN custom resource by performing PUT request to the pre-signed S3 URL.

import json
import requests

def lambda_handler(event, context):
    responseStatus = 'SUCCESS'
    responseData = {}
    if event['RequestType'] == 'Delete':
        sendResponse(event, context, responseStatus, responseData)

    responseData = {'Success': 'Test Passed.'}
    sendResponse(event, context, responseStatus, responseData)

def sendResponse(event, context, responseStatus, responseData):
    responseBody = {'Status': responseStatus,
                    'Reason': 'See the details in CloudWatch Log Stream: ' + context.log_stream_name,
                    'PhysicalResourceId': context.log_stream_name,
                    'StackId': event['StackId'],
                    'RequestId': event['RequestId'],
                    'LogicalResourceId': event['LogicalResourceId'],
                    'Data': responseData}
    print 'RESPONSE BODY:n' + json.dumps(responseBody)
    try:
        req = requests.put(event['ResponseURL'], data=json.dumps(responseBody))
        if req.status_code != 200:
            print req.text
            raise Exception('Recieved non 200 response while sending response to CFN.')
        return
    except requests.exceptions.RequestException as e:
        print e
        raise

if __name__ == '__main__':
    lambda_handler('event', 'handler')

I’m using json module to form the response body, json module is available by default in the Lambda container. Whereas the requests module which I use to perform the http PUT request needs to be packaged along with the python script.

Continuous Integration - Manage shared resources across accounts automatically

Mon, 07 Dec 2015 18:28:16 +0000

I prefer creating the base AMI using packer; RDS snapshot using a Jenkins job which get triggered whenever there is a change in the database schema in the SCM. In case of installers, binaries, etc. it would be best to store them in a single S3 bucket.

In all the above mentioned scenarios I prefer to share those resources to a 3rd party AWS account and remove the share when not needed. Automating this would be easy to handle add/remove permissions, hence I decided to hold the sharing details in a JSON file in the SCM. Changes to that file will trigger a Jenkins job which will invoke a Lambda function which in turn will share the resources based on the values in the JSON file.

Executing Ruby code in AWS Lambda

Sun, 29 Nov 2015 18:41:59 +0000

AWS Lambda supports only Java, JavaScript & Python at the time of writing this post. I was wondering whether I can exploit the Lambda to execute chef’s knife commands on it since Lambda runs on a Linux container. Using simple python function I have identified that the code of the Lambda function gets copied to /var/task and executed from there.

I decided to custom build ruby as embedded platform similar to how chef-client or chefdk are distributed. My requirement is to execute knife commands and its uses embedded ruby version 2.1.0, hence I downloaded the source archive that specific version.

Automated cross account DNS management through CFN and API access through API gateway

Fri, 27 Nov 2015 09:57:13 +0000

It’s a general best practice to manage all DNS entries in Route53 in a centralized AWS account. In that case it is difficult to automate the DNS record creation/deletion based on resources created in another AWS account using CloudFormation. CloudFormation doesn’t yet have the capability to create resources in a different AWS account.

Combining IAM role delegation, AWS Lambda & CFN Custom resources provides us a solution. I have extended the same solution by exposing the Lambda function through API gateway which provides a powerful mechanism. The illustration give below explains the solution.

Obtain AMI Id from name to be used in CFN

Sun, 22 Nov 2015 17:41:24 +0000

In most cases we create custom AMI’s for various reasons like OS hardening, installing and configuring additional software, etc. If you use single AWS account to create the AMI’s and share it with other AWS accounts and use that as part of CloudFormation template, it is required to pass the new AMI Id every time.

AWS Lambda comes handy for easy and elegant solution to get the latest AMI Id based on the AMI name and owner.

Save AWS costs by scheduled start and stop of EC2 instances

Sun, 15 Nov 2015 14:56:57 +0000

Most of the AWS resources are billed on per-hour basis which provides us an opportunity to save cost based on the usage pattern. Especially in case of Dev/Test environment, we need them only during the working days and working hours.

By using AWS Lambda function (python) in combination with EC2 instance tags, scheduled start or stop can be achieved with few lines of code.

Advantages of using AWS Lambda:

For 128MB memory we get 3,200,000 seconds free tier usage per month in AWS Lambda which will be more than enough for this function.
IAM role will be associated with the Lambda function which will be used to perform the required task and no separate keys need to be configured.

I used a tag with name as schedule & JSON string for tag value using which I could define the day and time when it needs to be started or stopped.