AWS Glue is a fully managed ETL service that makes it simple and cost-effective to categorize your data, clean it, enrich it, and move it reliably between various data stores. Glue job is the business logic that automate the extract, transform, and transfer data to different locations. Glue job creates a Hadoop task in the backend and since it uses Hadoop ecosystem all the nodes in the cluster must be able to find using the FQDN and communicate with each other. Read more

AWS Trusted Advisor is an important tool which provides more than 50 checks across four different categories Cost Optimization, Performance, Security & Fault Tolerance (at the moment of writing this post). But I have noticed that this is not being used more extensively considering the importance of it. Hence I created couple of lambda functions which shall be configured to run on daily basis to refresh the checks and create summary of all the checks and send an email. Read more

EBS snapshot as backup mechanism is a very common practice and it also has been defined in Backup and Recovery section of EC2 best practices. If you have read my another article Save AWS costs by scheduled start and stop of EC2 instances, similar approach will be used in creating Snapshots too. Using JSON in EC2 Tags enables us to provide granular configuration details. Create an EC2 tag with name as backup and value with details containing time to take backup and retention period. Read more

Cross-Region Replication for Amazon S3 was introduced last year which enables replicating objects from a S3 bucket to a different S3 bucket located in different region (it can be same/different AWS account). It’s one of the most sought after feature for long time but it doesn’t solve some of the scenarios as defined below. Synchronizing to a bucket in same region. Synchronizing to multiple destination buckets. Object ACLs needs to be set at the source bucket hence multiple different ACLs needs to be set when copying across different accounts. Read more

Windows Server 2016 Technical Preview 5 was released on 27th April 2016 & it’s well known by now that it supports docker with its windows server containers. Microsoft also started uploading windows server core with various application as docker images to docker hub. I was wondering whether it’s possible to run Windows 2016 TP5 on an EC2 instance and launch a windows container inside the EC2 instance. EC2 officially supports only upto Windows 2012 R2 and I couldn’t find any 2016 AMI’s in Ireland region. Read more