Continuous Integration – Manage shared resources across accounts automatically

I prefer creating the base AMI using packer; RDS snapshot using a Jenkins job which get triggered whenever there is a change in the database schema in the SCM. In case of installers, binaries, etc. it would be best to store them in a single S3 bucket.

In all the above mentioned scenarios I prefer to share those resources to a 3rd party AWS account and remove the share when not needed. Automating this would be easy to handle add/remove permissions, hence I decided to hold the sharing details in a JSON file in the SCM. Changes to that file will trigger a Jenkins job which will invoke a Lambda function which in turn will share the resources based on the values in the JSON file.

Sample JSON File

If I want to add a new account or remove permission to an existing account I just need to add/remove the specific account from the appropriate section in the JSON file and check-in to SCM. It will automatically trigger the Lambda function which performs the required changes. If you notice, in the above JSON file, for AMI & RDS names I’m just using the part of the string and not using the exact version. Hence it will find AMIs or RDS snapshots which contain this string in their name and modify the permission to all the resources. Share resources Jenkins job should also be triggered by (downstream job) the AMI creation / RDS Snapshot job, which ensures that all the new AMI/snapshot are shared appropriately, immediately after its creation.

Lambda Function to share resources

At the time of writing this blog, default boto3 version in AWS Lambda is 1.2.1 whereas RDS snapshot sharing API is available only in version 1.2.2. Hence I had to pack boto3 also as part of the Lambda function. Best way to include boto3 module is to download it to local directory using pip.

It will download boto3 and its dependent modules to the local directory named ShareAWSResources which should be packed in a zip file including the python script.

IAM permissions required for the Lambda function

Leave a Comment

Your email address will not be published.

1 Trackback

  1. AWS Week in Review – December 7, 2015 | SMACBUZZ (Pingback)