December 7, 2015

Continuous Integration – Manage shared resources across accounts automatically

I prefer creating the base AMI using packer; RDS snapshot using a Jenkins job which get triggered whenever there is a change in the database schema in the SCM. In case of installers, binaries, etc. it would be best to store them in a single S3 bucket.

In all the above mentioned scenarios I prefer to share those resources to a 3rd party AWS account and remove the share when not needed. Automating this would be easy to handle add/remove permissions, hence I decided to hold the sharing details in a JSON file in the SCM. Changes to that file will trigger a Jenkins job which will invoke a Lambda function which in turn will share the resources based on the values in the JSON file.

► Sample JSON File
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
{
  "ami": {
    "base_linux_v": {
      "eu-west-1": [
        "123456789012",
        "987654321098"
      ],
      "us-east-1": [
        "987654321098"
      ]
    },
    "base_windows_v": {
      "eu-west-1": [
        "123456789012",
        "987654321098"
      ],
      "us-east-1": [
        "123456789012"
      ]
    }
  },
  "s3": {
    "com-wordpress-cloudenlightened-blog": [
      "123456789012",
      "987654321098"
    ]
  },
  "rds": {
    "base-db-v": {
      "eu-west-1": [
        "123456789012",
        "987654321098"
      ],
      "us-east-1": [
        "987654321098"
      ]
    }
  }
}

If I want to add a new account or remove permission to an existing account I just need to add/remove the specific account from the appropriate section in the JSON file and check-in to SCM. It will automatically trigger the Lambda function which performs the required changes. If you notice, in the above JSON file, for AMI & RDS names I’m just using the part of the string and not using the exact version. Hence it will find AMIs or RDS snapshots which contain this string in their name and modify the permission to all the resources. Share resources Jenkins job should also be triggered by (downstream job) the AMI creation / RDS Snapshot job, which ensures that all the new AMI/snapshot are shared appropriately, immediately after its creation.

► Lambda Function to share resources
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
import boto3
import sys

#
# Share the AWS resources based on the data received from the event.
#
def lambda_handler(event, context):
    print "=== Started aws resources sharing ==="
    if event['ami']:
        share_amis(event['ami'])
    if event['s3']:
        share_s3_bucket(event['s3'])
    if event['rds']:
        share_rds_snapshot(event['rds'])


def share_amis(amitree):
    for amis in amitree:
        for reg in amitree[amis]:
            desaccess = amitree[amis][reg]
            ec2 = boto3.client('ec2', region_name=reg)
            amilist = get_ami_ids(ec2, amis)
            for amiid in amilist:
                exstaccess = get_ami_perms(ec2, amiid)
                addacct = list(set(desaccess) - set(exstaccess))
                remacct = list(set(exstaccess) - set(desaccess))
                modify_ami_perms(ec2, amiid, addacct, remacct)


def get_ami_ids(ec2, aminame):
    print 'Getting list of AMIs with name ' + aminame
    amiids = []
    try:
        amiout = ec2.describe_images(Filters=[{'Name': 'name', 'Values': [aminame + '*']}])
    except Exception as e:
        print "Error: %s" % e
        sys.exit(1)

    for img in amiout['Images']:
        amiids.append(img['ImageId'])
    return amiids


def get_ami_perms(ec2, amiid):
    print 'Getting launch permissions of AMI ID ' + amiid
    acctids = []
    try:
        acctout = ec2.describe_image_attribute(ImageId=amiid, Attribute='launchPermission')
    except Exception as e:
        print "Error: %s" % e
        sys.exit(1)

    for acct in acctout['LaunchPermissions']:
        acctids.append(acct['UserId'])
    return acctids


def modify_ami_perms(ec2, amiid, addacct, remacct):
    print 'Modifying permissions for AMI ID ' + amiid
    addcoll = []
    remcoll = []
    for acct in addacct:
        addcoll.append({'UserId': acct})
    for acct in remacct:
        remcoll.append({'UserId': acct})
    try:
        if addcoll:
            ec2.modify_image_attribute(ImageId=amiid, Attribute='launchPermission',
                                       LaunchPermission={'Add': addcoll})
        if remcoll:
            ec2.modify_image_attribute(ImageId=amiid, Attribute='launchPermission',
                                       LaunchPermission={'Remove': remcoll})
    except Exception as e:
        print "Error: %s" % e
        sys.exit(1)


def share_rds_snapshot(rdstree):
    for snap in rdstree:
        for reg in rdstree[snap]:
            desaccess = rdstree[snap][reg]
            rds = boto3.client('rds', region_name=reg)
            snaplist = [s for s in get_all_snapshots(rds) if snap in s]
            for sn in snaplist:
                exstaccess = get_snap_perms(rds, sn)
                addacct = list(set(desaccess) - set(exstaccess))
                remacct = list(set(exstaccess) - set(desaccess))
                modify_snapshot_perms(rds, sn, addacct, remacct)


def get_all_snapshots(rds):
    try:
        snapout = rds.describe_db_snapshots()
        snaplist = []
        for snap in snapout['DBSnapshots']:
            snaplist.append(snap['DBSnapshotIdentifier'])
        return snaplist
    except Exception as e:
        print "Error: %s" % e
        sys.exit(1)


def get_snap_perms(rds, snapname):
    print 'Getting existing permission to snapshot ' + snapname
    acctids = []
    try:
        acctout = rds.describe_db_snapshot_attributes(DBSnapshotIdentifier=snapname)
    except Exception as e:
        print "Error: %s" % e
        sys.exit(1)

    for acct in acctout['DBSnapshotAttributesResult']['DBSnapshotAttributes']:
        print acct
        if acct['AttributeName'] == 'restore':
            acctids = acct['AttributeValues']
    return acctids


def modify_snapshot_perms(rds, snap, addacct, remacct):
    print 'Modifying permissions for snapshot ' + snap
    try:
        if addacct:
            rds.modify_db_snapshot_attribute(DBSnapshotIdentifier=snap, AttributeName='restore', ValuesToAdd=addacct)
        if remacct:
            rds.modify_db_snapshot_attribute(DBSnapshotIdentifier=snap, AttributeName='restore', ValuesToRemove=remacct)
    except Exception as e:
        print "Error: %s" % e
        sys.exit(1)


def share_s3_bucket(s3bucks):
    s3 = boto3.client('s3')
    for buck in s3bucks:
        set_s3_bucket_policy(s3, buck, s3bucks[buck])


def set_s3_bucket_policy(s3, buck, acctids):
    print 'Setting policy to S3 bucket ' + buck
    if acctids:
        policy = '{ "Version": "2012-10-17", "Id": "ReadBucketPolicy", "Statement": [ { "Sid": "S3ReadAccessToRole",
                  "Effect": "Allow", "Principal": { "AWS": ['
        for acct in acctids[:-1]:
            policy += '"arn:aws:iam::' + acct + ':root",'
        else:
            policy += '"arn:aws:iam::' + acctids[-1] + ':root"] }, "Action": [ "s3:List*", "s3:Get*" ], "Resource": [
                       "arn:aws:s3:::' + buck + '/*","arn:aws:s3:::' + buck + '" ] } ] }'
    try:
        if acctids:
            s3.put_bucket_policy(Bucket=buck, Policy=policy)
        else:
            s3.delete_bucket_policy(Bucket=buck)
    except Exception as e:
        print "Error: %s" % e
        sys.exit(1)


if __name__ == '__main__':
    lambda_handler('event', 'handler')

At the time of writing this blog, default boto3 version in AWS Lambda is 1.2.1 whereas RDS snapshot sharing API is available only in version 1.2.2. Hence I had to pack boto3 also as part of the Lambda function. Best way to include boto3 module is to download it to local directory using pip.

pip install boto3 –t ./ShareAWSResources

It will download boto3 and its dependent modules to the local directory named ShareAWSResources which should be packed in a zip file including the python script.

► IAM permissions required for the Lambda function
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AMIPolicy",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeImageAttribute",
                "ec2:DescribeImages",
                "ec2:ModifyImageAttribute"
            ],
            "Resource": [
                "*"
            ]
        },
		{
            "Sid": "S3Policy",
            "Effect": "Allow",
            "Action": [
                "s3:PutBucketPolicy",
                "s3:DeleteBucketPolicy"
            ],
            "Resource": [
                "*"
            ]
        },
		{
            "Sid": "RDSSnapshotPolicy",
            "Effect": "Allow",
            "Action": [
                "rds:DescribeDBSnapshots",
                "rds:DescribeDBSnapshotAttributes",
                "rds:ModifyDBSnapshotAttribute"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

© Prakash P 2015 - 2019

Powered by Hugo & Kiss.