Scheduling automated EBS snapshots serverless using AWS Lambda

EBS snapshot as backup mechanism is a very common practice and it also has been defined in Backup and Recovery section of EC2 best practices. If you have read my another article Save AWS costs by scheduled start and stop of EC2 instances, similar approach will be used in creating Snapshots too.

Using JSON in EC2 Tags enables us to provide granular configuration details. Create an EC2 tag with name as backup and value with details containing time to take backup and retention period. Obtaining these details from Tags will enable us to configure different schedule & retention period for different EC2 instance. If no backup is needed on a particular day, that day shall be removed from the value.

Sample tag value

Lambda function

This function contains a config section which defines how you expect the script to behave. If auto-create-tag is set to true the script will check all EC2 instances for a tag named backup, if it exists it will use the value of it. If it doesn’t exists, it will create the tag with the value as defined in default section of config. Script can be configured to ignore certain EC2 instances by specifying their instance ID’s in exclude or the name of the instance in exclude_name section. If an EC2 instance contains a tag named ignore the script will ignore that instance too.

If auto-create-tag is set to false the script will check all EC2 instances for a tag named backup, if it exists it will trigger backup, else ignore that instance.

Once the script identifies that the instance needs to be backed up, it will get the list of EBS volumes attached to that instance and create snapshot of them. Once snapshot creation has been triggered, it will check the number of snapshots created from that volume and delete the older snapshots to maintain the retention period configured in the tags.

If sns_topic has been configured with ARN of the SNS topic, it will send an email notification whenever it fails to create the snapshot.

List of privileges that should be enabled for the IAM role associated to the Lambda function.

Lambda function shall be scheduled to run every hour.

Lambda configuration:
Add scheduler as event source and configure it to run every hour.

Lambda_Schedule_Event

8 Comments

 Add your comment
  1. Hi Prakash,
    You have posted very good blogs and here i would like to now how we can programming for creating lambda fucntion it looks to me very challenging task. I am not from programming background could you please give some suggestion here.

  2. Hi,

    I’m trying to get your Blog Entry Scheduling automated EBS snapshots serverless using AWS Lambda ready to work. Unfortunately I get he following error

    Syntax error in module lambda_function: invalid syntax (lambda_function.py, line 14)

    Could you give some support?

    BR

    René

  3. Hi,

    I’m trying to get your Blog Entry “Scheduling automated EBS snapshots serverless using AWS Lambda” ready to work. Unfortunately I get he following error

    Syntax error in module ‘lambda_function’: invalid syntax (lambda_function.py, line 14)

    Could you give some support?

    BR

    René

  4. Hi,

    ich have the following error: Syntax error in module ‘lambda_function’: invalid syntax (lambda_function.py, line 14)

    any idea on that

  5. There are couple of issues which I came across when I tried to use Lambda as backup solution:
    1. Lambda is limited to 5 mins. of running time and snapshot might take different length of time to complete depending on many factors (amount of changed data, etc. ) so we don’t know for sure if it was successful or not. It can take from 1-2 minutes to 40 minutes in my experience.
    2. Retention deletes old snapshot without knowing that the current snapshot has been successfully completed.

    Regards,
    Constantin

  6. Nice article. I think maintaining this will soon become cumbersome though, once you start using a lot of instances. Tags don’t have version control for one thing…

  7. Nice article. But maintaining backup policies in EC2 tags will become cumbersome fast when the number of instances increases?

Leave a Comment

Your email address will not be published.