Case Study – WIP Reporting and Timeline video on Completion

Requirements (My hallucinations):

Design and architect a highly available, large user base system which is going to be used by the National Highways, the regular employees updating photos of WIP on different stages and when work completed, archive all images after creating a timeline video. WIP sequence should keep the most latest photo thumbnail linked to a project blog page, with a gallery linking to last photo per day.  Post-processing of a completed work can take even up to a week giving more importance to the lowest cost possible. The system should be capable of handling hundreds of thousands of high-quality mobile photographs per day. Runtime costs should be as low as possible. For each WIP a minimum of one photograph in six hours is desired. 

Solution on AWS (My views):

Application to be developed in some kind of single-page-app with progressive-web-app support, javascript and CSS libraries. This can be hosted on AWS S3 bucket with Cloud Front default origin pointed here. The standard secure approach of https (Redirect HTTP to HTTPS), OAI and custom domain with certificate from ACM is recommended. Dynamic part uses Cognito User Pool, Amazon API Gateway (regional), Lambda, STS etc. API Gateway stage should be the behaviour point for route.

Continue reading “Case Study – WIP Reporting and Timeline video on Completion”

Export Cloudwatch Logs to AWS S3 – Deploy using SAM

With due reference to the blog which helped me in the right direction, the Tensult blogs article Exporting of AWS CloudWatch logs to S3 using Automation, though at some points I have deviated from the original author’s suggestion.

Some points are blindly my preference and some other due to the suggested best practices. I do agree that starters, would be better off with setting IAM policies with ‘*’ in the resource field. But when you move things into production it is recommended to use least required permissions. Also, some critical policies were missing from the assume role policy. Another unnecessary activity was the checking of the existence of s3 bucket and attempt to create if not exists, at each repeated execution. Again for this purpose the lambda role needed create bucket permission. All these were over my head, and the outcome is this article.

Well if you need CloudWatch logs to be exported to S3 for whatever reason, this could save your time a lot, though this needs to be run in every different region where you need to deploy the stack. Please excuse me as the whole article expects to have aws-cli and sam-cli pre-installed.

Continue reading “Export Cloudwatch Logs to AWS S3 – Deploy using SAM”