Refactored a Complicated Lambda to use Layers and split it up

Till recently, in fact, till last week, was not too worried about writing all code into single code folder, and mapping multiple AWS::Serverless::Function into individual named handlers. Till I stumbled on this article, where I started wondering how my folder structure and sam templates were going into the stack. A detailed inspection was not required, though this was the time when I used the GUI ( after a long time ). But the outcome showed how pathetic the condition was.

The lambda console with the filter “aws:cloudformation:stack-name: <stack>”

Well, it is clear that the whole mess is being uploaded into all the function code. What does this mean – holy grail, any one small change here or there, would update all the functions – last modified is the same, all functions will have the node_modules and other artifacts like templates and custom modules.

Well I started today to do the optimization, armed with the credits earned from the AWS Community Builder Program, duplicated the codebase to another folder, and started the refactoring. I might have copied the template code from here, which explains my layer with a different description in the screenshot. It took me about 3 or 4 layer versions to strike the correct structure.

Starting with lib/ then adding a nodejs/ beneath that, and finally moving node_modules into this folder. The structure is simple that our layer should have the runtime folder in which we can have the modules. Now was confused about how to make my custom modules, which were not available on any registry, to be available for the handlers to import using require. Just thought that by creating a folder inside node_modules and putting all my custom modules into that should work. Well, it worked, though some relative paths started breaking. Finally these also were resolved by using /opt/nodejs/node_modules/custom-module/ instead of ./

The listing is sorted in the same order as the original, see the size difference

Basically, the size difference is one big advantage, and when we run deploy for a minor change in any handler, instead of packing the whole whopping 500kb, the single handler is zipped and uploaded for the deployment. Also change in one handler will not affect the container version for other functions which could be a better tradeoff for heavily active applications taking advantage of reuse of active containers.

Now considering to add one more layer with our custom modules, since these can get evolved and require slight modifications during further development. Currently since these are along with the main layer, changes could bring in the upload of the whole node_modules folder.

After a round of smoke testing, the whole codebase was migrated back into the production system and deployed there. Thereafter deploy was completed, a full round of functional tests was run.