Executing Ruby code in AWS Lambda

AWS Lambda supports only Java, JavaScript & Python at the time of writing this post. I was wondering whether I can exploit the Lambda to execute chef’s knife commands on it since Lambda runs on a Linux container. Using simple python function I have identified that the code of the Lambda function gets copied to /var/task and executed from there.

I decided to custom build ruby as embedded platform similar to how chef-client or chefdk are distributed. My requirement is to execute knife commands and its uses embedded ruby version 2.1.0, hence I downloaded the source archive that specific version.

Lambda have certain limitations on the package size that the total uncompressed size of code and its dependencies should not exceed 250MB. Due to this limitation I compiled ruby by configuring it with limited set of options as listed below.

Note: Remember to install zlib, zlib-devel, openssl & openssl-devel before compiling ruby for gem installation and ssl access.

Once the ruby has been compiled and installed, ruby binaries will be available under /var/task/customruby/bin/. Using the gem binary under this location, install all the required gem files.

In my case I installed all the gems which are part of the chef client from previously installed local gem cache with –no-ri & –no-rdoc to save space in the package. Created a directory containing knife.rb & validator key which are needed to connect to the chef server. I created a simple python program to validate it, which lists all the environments in the chef server.

Lambda function in action:

LambdaRuby_log

This opens up whole new opportunity and I don’t need to wait for AWS to provide support for ruby, at least for my requirement.

10 Comments

 Add your comment
  1. Hi Prakash,

    Just wanted to thank you again for this. Your post got me over the hump of getting Ruby bundled with our Lambda function which now powers our automated Jekyll website builds. If you ever have any interest in porting cloudenlightened from WordPress blog to Jekyll, have a look at http://www.aerobatic.com. Free plan comes with a custom domain and SSL. Here’s the blog post on the announcement: https://www.aerobatic.com/blog/automated-continuous-deployment-of-jekyll-sites.

    Regards,
    David

    • Hi David,

      Happy to hear that my post helped you.

      That offer from aerobatic looks interesting, sure I’ll look at it when I plan to move out of WordPress.

      Cheers,
      Prakash

  2. Hi Prakash,

    Thanks for the response. After building ruby on my Linux AMI the ruby-2.3.0 directory is 370MB. I’m assuming I don’t bring down the entire thing to bundle in the Lambda zip. The ruby binary is there as is a bin directory with gem, but then there’s a bunch of other stuff including .c files and many directories such as “sample”, “spec”, “coverage”, “cygwin” and so on. What I’m trying to figure out is what the contents of the customruby directory should look like.

    Thanks in advance,
    David

  3. Hi, this is exactly what I’ve been looking for. Wonder if you could “enlighten” me with a few more details on how you deployed. I’ve compiled a ruby binary on Amazon Linux which I can certainly include it in the zip file that gets deployed to Lambda, but I’m trying to understand this paragraph:

    “Once the ruby has been compiled and installed, ruby binaries will be available under /var/task/customruby/bin/. Using the gem binary under this location, install all the required gem files.”

    Is this done on the Amazon Linux box where ruby was built from source? If that’s the case, what files do I need to scp locally and include in the Lambda zip? The entire contents of the /var/task/customruby directory?

    Sorry if this should be obvious, building from raw source is not something I’m very familiar with.

    Thanks for sharing this valuable info.

    • Hi David,

      If you want to install any specific gem files you need to use the gem binary located in /var/task/customruby/bin/ hence it will get installed in the appropriate location inside that directory. Once you have all ruby stuffs in place you need to create the python script which will be executed by lambda function.

      Yes, it was built in Amazon Linux. You should zip the python script and customruby directory together, similar to following structure.

      exceuteruby.zip
      |
      exceuteruby.py
      |
      customruby

      Do let me know if you need any additional details.

  4. Nice Article.
    What is the impact on performance with ruby running as embedded code?

    • I’m not running a performance intensive executions yet. Simple knife list command took 7 seconds. knife usually takes few seconds even in the native ruby platform.

      REPORT RequestId: 7ad47b68-96c4-11e5-ad5d-892aff3a4866 Duration: 6806.93 ms Billed Duration: 6900 ms Memory Size: 128 MB Max Memory Used: 66 MB

  5. Awesome article thanks. On which environment do you compile the rubies and install the gems? Is this in an EC2 instance? Which distro? Thanks!

Leave a Comment

Your email address will not be published.

2 Trackbacks

  1. AWS Week in Review – November 30, 2015 | wart1949 (Pingback)
  2. AWS Week in Review – November 23, 2015 | wart1949 (Pingback)