AWS – Page 2 – JijuTM.COM

Choosing the Right Database for High-Performance Web Applications on AWS

In any web application project, selecting the optimal database is crucial. Each project comes with unique requirements, and the final decision often depends on the data characteristics, the application’s operational demands, and future scaling expectations. For my most recent project, choosing a database meant evaluating a range of engines, each with strengths and trade-offs. Here, I’ll walk through the decision-making process and the architecture chosen to meet the application’s unique needs using AWS services.

Initial Considerations

When evaluating databases, I focused on several key factors:

Data Ingestion and Retrieval Patterns: What type of data will be stored, and how will it be accessed or analyzed?
Search and Select Complexity: How complex are the queries, and do we require complex joins or aggregations?
Data Analysis Needs: Will the data require post-processing or machine learning integration for tasks like sentiment analysis?

The database engines I considered included MariaDB, PostgreSQL, and Amazon DynamoDB. MariaDB and PostgreSQL are widely adopted relational databases known for reliability and extensive features, but DynamoDB is particularly designed to support high-throughput applications on AWS, making it a strong candidate.

The Project’s Data Requirements

This project required the following data structure:

Data Structure: Each row was structured as JSON, with a maximum record size of approximately 1,541 bytes.
Attributes: Each record included an asset ID (20 chars), user ID (20 chars), a rating (1 digit), and a review of up to 1,500 characters.
Scale Expectations: Marketing projections suggested rapid growth, with up to 100,000 assets and 50,000 users within six months, resulting in a peak usage of about 5,000 transactions per second. Mock Benchmarks and Testing

To ensure scalability, I conducted a benchmarking exercise using Docker containers to simulate real-world performance for each database engine:

MariaDB and PostgreSQL: Both performed well with moderate loads, but resource consumption spiked sharply under simultaneous requests, capping at around 50 transactions per second before exhausting resources.
Amazon DynamoDB: Even on constrained resources, DynamoDB managed up to 24,000 requests per second. This performance, combined with its fully managed, serverless nature and built-in horizontal scaling capability, made DynamoDB the clear choice for this project’s high concurrency and low-latency requirements. Amazon DynamoDB – The Core Database

DynamoDB emerged as the best fit for several reasons:

High Availability and Scalability: With DynamoDB, we can automatically scale up or down based on traffic, and AWS manages the underlying infrastructure, ensuring availability across multiple regions.
Serverless Architecture Compatibility: Since our application was API-first and serverless, built with AWS Lambda in Node.js and Python, DynamoDB’s seamless integration with AWS services suited this architecture perfectly.
Flexible Data Model: DynamoDB’s schema-less, JSON-compatible structure aligned with our data requirements.

Automating Laptop Charging with AWS: A Smart Solution to Prevent Overheating

In today’s fast-paced digital world, laptops have become indispensable tools. However, excessive charging can lead to overheating, which can significantly impact performance and battery life. In this blog post, we’ll explore a smart solution that leverages AWS services to automate laptop charging, prevent overheating, and optimize battery health. I do agree that Asus does provide premium support for a subscription, but this research and excercise was to brush up my brains and learn to create on aws with some useful solution. The solution is still in concept and once I start using it in production to the full extend, the shell scripts and cloudformation template will be pushed into github handle jthoma repository code-collection/aws

Understanding the Problem:

Overcharging can cause the battery to degrade faster and generate excessive heat. Traditional manual charging methods often lead to inconsistent charging patterns, potentially harming the battery’s lifespan.

The Solution: Automating Laptop Charging with AWS

To address this issue, we’ll utilize a combination of AWS services to create a robust and efficient automated charging system:

AWS IoT Core: Purpose: This service enables secure and reliable bi-directional communication between devices and the cloud.
How it’s used: We’ll connect a smart power outlet to AWS IoT Core, allowing it to send real-time battery level data to the cloud.
Link: https://aws.amazon.com/iot-core/
Getting Started: Sign up for an AWS account and create an IoT Core project.
AWS Lambda: Purpose: This serverless computing service allows you to run code without provisioning or managing servers.
How it’s used: We’ll create a Lambda function triggered by IoT Core messages. This function will analyze the battery level and determine whether to charge or disconnect the power supply.
Link: https://aws.amazon.com/lambda/
Getting Started: Create a Lambda function and write the necessary code in your preferred language (e.g., Python, Node.js, Java).
Amazon DynamoDB: Purpose: This fully managed NoSQL database service offers fast and predictable performance with seamless scalability.
Link: https://aws.amazon.com/dynamodb/
Amazon CloudWatch: Purpose: This monitoring and logging service helps you collect and analyze system and application performance metrics.
How it’s used: We’ll use CloudWatch to log system health and generate alarms based on battery level or temperature threshold. Also it helps to monitor the performance of our Lambda functions and IoT Core devices, ensuring optimal system health.
Link: https://aws.amazon.com/cloudwatch/
Getting Started: Configure CloudWatch to monitor your AWS resources and set up alarms for critical events.

How it Works:

Data Collection: My Ubuntu system with the help of a shell script uses aws cli to send real-time battery level data to the cloud watch logs.
Data Processing: Cloud watch metric filter alarms will trigger lambda function which is set for appropriate actions.
Action Execution: The Lambda function sends commands to the smart power outlet to control the charging process.
Data Storage: Historical battery level data is stored in Cloud Watch logs for analysis using Athena and further optimization.
Monitoring and Alerting: CloudWatch monitors the system’s health and sends alerts if any issues arise.

Benefits of Automated Charging:

Optimized Battery Health: Prevents overcharging and undercharging, extending battery life.
Reduced Heat Generation: Minimizes thermal stress on the laptop.
Improved Performance: Ensures optimal battery performance, leading to better system responsiveness.
Energy Efficiency: Reduces energy consumption by avoiding unnecessary charging.

Conclusion

By leveraging AWS services, a sophisticated automated charging system that safeguards the laptop’s battery health and enhances its overall performance is reached. This solution empowers you to take control of your device’s charging habits and enjoy a longer-lasting, cooler, and more efficient laptop.

Start Your AWS Journey Today, Signup for free !

Ready to embark on your cloud journey? Sign up for an AWS account and explore the vast possibilities of cloud computing. With AWS, you can build innovative solutions and transform your business.

Amazon Q Developer: A Generative AI-Powered Conversational Assistant for Developers

Amazon Q Developer is a generative artificial intelligence (AI) powered conversational assistant designed to support developers in understanding, building, extending, and managing AWS applications. By leveraging the power of generative AI, Amazon Q Developer can provide developers with a variety of benefits, including:

Enhanced Understanding: Developers can ask questions about AWS architecture, resources, best practices, documentation, support, and more. Amazon Q Developer provides clear and concise answers, helping developers quickly grasp complex concepts.
Accelerated Development: Amazon Q Developer can assist in writing code, suggesting improvements, and automating repetitive tasks. This can significantly boost developer productivity and efficiency.
Improved Code Quality: By identifying potential issues and suggesting optimizations, Amazon Q Developer helps developers write cleaner, more secure, and more reliable code.

Amazon Q Developer is powered by Amazon Bedrock, a fully managed service that provides access to various foundation models (FMs). The model powering Amazon Q Developer has been specifically trained on high-quality AWS content, ensuring developers receive accurate and relevant answers to their questions.

Key Features of Amazon Q Developer:

Conversational Interface: Interact with Amazon Q Developer through a natural language interface, allowing easy and intuitive communication.
Code Generation and Completion: Receive code suggestions and completions as you type, reducing the time spent writing code.
Code Review and Optimization: Identify potential issues in your code and receive recommendations for improvements.
AWS-Specific Knowledge: Access a wealth of information about AWS services, best practices, and troubleshooting tips.
Continuous Learning: Amazon Q Developer is constantly learning and improving, ensuring that you always have access to the latest information.

How to Get Started with Amazon Q Developer:

Sign up for an AWS account: If you don’t already have one, create an AWS account to access Amazon Q Developer.
Install the Amazon Q Developer extension: Download and install the Amazon Q Developer extension for your preferred IDE (e.g., Visual Studio Code).
Start asking questions: Begin interacting with Amazon Q Developer by asking questions about AWS, your code, or specific development tasks.

By leveraging the power of generative AI, Amazon Q Developer empowers developers to work more efficiently, write better code, and accelerate their development process.

Built a Feature-Rich QR Code Generator with Generative AI and JavaScript

In today’s digital world, QR codes have become ubiquitous. From restaurant menus to product packaging, these scannable squares offer a convenient way to access information. This article details the creation of a versatile QR code generator that leverages the power of generative AI and JavaScript for a seamless user experience, all within the user’s environment.

Empowering Development with Generative AI

The project began by utilizing generative AI tools to generate boilerplate code. This innovative approach demonstrates the potential of AI to streamline development processes. Prompts are used to create a foundation, allowing developers to focus on implementing advanced functionalities.

Generative AI Coding primer

Open Google Gemini and type the following

Assume the role of a HTML coding expert <enter>

Watch for the response, and if it is positive, go ahead and continue to tell it what you want. Actually for this project the next prompt I gave was:

Show me an HTML boiler plate starter with Bootstrap and JQquery linked from public cdn libraries.

Then for each element, the correct description was suggested, like adding form, text input, further reset button, submit button, and download button initially hidden. The rest of the functionality was very easy with qrcodejs library and further new chat with role setting.

Assume the role of a JavaScript programmer with hefty JQuery experience.

Further prompts were curated to get the whole builder ready still I had to use a bit of my expertise and commonsense, while local testing was done using the node js utility HTTP-server which was installed with Gemini’s suggested command.

prompt:

node http server install

from the response:

npm install http-server -g

Key Functionalities

The QR code generator boasts several user-friendly features, all processed entirely on the client-side (user’s device):

Phone Number Validation and WhatsApp Integration:
- Users can input phone numbers, and the code validates them using regular expressions.
- Validated numbers are converted into WhatsApp direct chat links, eliminating the need for external servers and simplifying communication initiation.
QR Code Generation for Phone Calls:
- The application generates QR codes that trigger phone calls when scanned by a mobile camera when provided with the proper intent URL. tel://<full mobile number>
- This is a practical solution for scenarios like displaying contact information on a car, without ever sending your phone number outside your device.

Technical Deep Dive

The project leverages the following technologies, emphasizing the client-side approach:

Client-Side Functionality with JavaScript:
- This eliminates the need for a server, making the application fast, efficient, and easy to deploy. Users experience no delays while generating QR codes, and all processing stays within their browser.
AWS S3 Website Delivery:
- Cost-effective and scalable hosting for the static website ensures smooth operation. S3 simply serves the application files, without any server-side processing of user data.
AWS CloudFront for Global Edge Caching and Free SSL:
- CloudFront enhances performance by caching static content closer to users globally, minimizing latency. Free SSL certification guarantees secure communication between users and your website, even though no user data is transmitted.
- Please visit review and comment on my QR Code Generator, the known bug in some mobile phones is the download fails, which I will see to as soon as possible, if that is the case with your phone take a screenshot and crop it up for the time being. On Samsung devices I think the power button and volume down pressed together would take a screenshot.

Unveiling the Cloud: A Recap of AWS Community Day Mumbai 2024

On April 6th, the Mumbai cloud community converged at The Lalit for AWS Community Day 2024. This electrifying one-day event, organized by the AWS User Group Mumbai, brought together enthusiasts from all walks of the cloud journey – from budding developers to seasoned architects.

A Day of Learning and Sharing

The atmosphere crackled with a shared passion for cloud technology. The agenda boasted a variety of sessions catering to diverse interests. Whether you were keen on optimizing multi-region architectures or building personalized GenAI applications, there was a talk designed to expand your knowledge base.

Workshops: Deep Dives into Specific Topics

For those seeking a more hands-on experience, workshops offered an invaluable opportunity to delve deeper into specific topics. Attendees with workshop passes could choose from two exciting options:

Lower latency of your multi-region architecture with Kubernetes, Couchbase, and Qovery on AWS: This workshop equipped participants with the know-how to optimize their multi-region deployments for minimal latency.
Create a personalised GenAI application with Snowflake, Streamlit and AWS Bedrock to cross-sell products: This session focused on building engaging GenAI applications that leverage the power of Snowflake, Streamlit, and AWS Bedrock to personalize the customer experience.

A Community of Builders

Beyond the technical learning, the true spirit of the event resided in the sense of community. The venue buzzed with conversations as attendees exchanged ideas, shared experiences, and built connections. This collaborative atmosphere fostered a valuable space for peer-to-peer learning and professional growth.

A Noteworthy Collaboration

The event was further enriched by the collaboration with Snowflake. Their insightful workshop on building personalized GenAI applications provided a unique perspective on leveraging cloud technologies for enhanced customer experiences.

A Day Well Spent

AWS Community Day Mumbai 2024 proved to be a resounding success. It offered a platform for attendees to gain valuable knowledge, explore the latest cloud innovations, and connect with a vibrant community. If you’re based in Mumbai and have a passion for cloud computing, attending the next AWS Community Day is a surefire way to elevate your skills and stay ahead of the curve.

Unleashing the Power of AWS DynamoDB: Exploring Common Use Cases

Amazon Web Services (AWS) DynamoDB stands tall as a powerful, fully managed NoSQL database service, offering seamless scalability, high availability, and low latency. Its flexibility and performance make it a favorite among developers and businesses across various industries. Let’s delve into some common use cases where DynamoDB shines brightly:

1. Real-Time Analytics: DynamoDB’s ability to handle massive volumes of data with lightning-fast response times makes it ideal for real-time analytics applications. Whether it’s tracking user interactions on a website, monitoring IoT devices, or analyzing streaming data, DynamoDB efficiently stores and retrieves data, enabling businesses to make data-driven decisions in real-time.

2. Ad Tech Platforms: Ad tech platforms deal with immense amounts of data generated from ad impressions, clicks, and user interactions. DynamoDB’s ability to handle high throughput and scale automatically makes it an excellent choice for storing and retrieving this data rapidly. Additionally, its integration with other AWS services like Lambda and Kinesis enables seamless data processing and analysis pipelines.

3. Gaming Leaderboards: Gaming applications often require storing and updating leaderboards in real-time to provide players with up-to-date rankings. DynamoDB’s fast read and write capabilities, along with its scalability, make it a perfect fit for maintaining dynamic leaderboards, ensuring a smooth and engaging gaming experience for players worldwide.

4. Content Management Systems (CMS): Content-heavy applications, such as CMS platforms and blogging websites, benefit from DynamoDB’s ability to handle large volumes of structured and unstructured data. Whether it’s storing user-generated content, managing metadata, or tracking user interactions, DynamoDB provides the scalability and performance required to deliver content quickly and reliably to users.

5. E-commerce Applications: DynamoDB plays a crucial role in e-commerce applications by efficiently managing product catalogs, customer profiles, and transaction data. Its seamless scalability ensures that e-commerce platforms can handle sudden spikes in traffic during peak shopping seasons, while its low latency guarantees a smooth shopping experience for customers browsing and purchasing products online.

6. Internet of Things (IoT) Data Management: IoT devices generate vast amounts of data that need to be collected, processed, and analyzed in real-time. DynamoDB’s ability to handle high throughput and store structured and semi-structured data makes it an ideal choice for managing IoT data streams. Whether it’s monitoring sensor data, tracking device status, or analyzing telemetry data, DynamoDB provides the scalability and performance required for IoT applications.

7. User Session Management: Applications that require managing user sessions, such as chat applications and collaborative platforms, can leverage DynamoDB to store session data securely and efficiently. DynamoDB’s fast read and write operations ensure quick access to session data, enabling seamless user experiences across multiple devices and sessions.

8. Financial Services: In the financial services sector, DynamoDB is used for various applications, including fraud detection, risk management, and transaction processing. Its ability to handle high volumes of data with low latency makes it well-suited for real-time financial analytics and compliance reporting, ensuring the security and reliability of financial transactions.

In conclusion, AWS DynamoDB offers a versatile and scalable solution for a wide range of use cases across industries. Whether it’s real-time analytics, gaming leaderboards, e-commerce applications, or IoT data management, DynamoDB empowers businesses to build high-performance, scalable, and reliable applications that deliver exceptional user experiences. As technology continues to evolve, DynamoDB remains at the forefront, driving innovation and enabling businesses to thrive in the digital age.

Attempt to create animated representation of AWS DevOps pipeline

Though the title says something technical this is just a self-promotion and cheap boasting

Continuing with the boosting as I have been doing this for the past couple of days. No, I am not insane, but wanted to do this by hand and use some shell commands. Initially the scenes were identified as 10 and folders created with a base flowchart made using Libre Office Draw copied into each of the folders.

Finally the full image sequence was copied into “full” with renaming in sequence with the following command.

Before that, the same command was previewed using echo instead of cp as seen below.

And finally all images were in the “full” folder as below.

It was time to invoke ffmpeg as shown below.

ffmpeg -i dop%04d.png -c:v libx264 -an -r 30 ../dop-anim.mp4 -hide_banner

What could have been achieved with paid tools like Canva or many others, with some effort and free tools available with Ubuntu Linux achieved with minimal expense, without considering my work time earnings that should be a concern.

When I went to SJCCD 2024

This is not a technical document but rather a place to show off some pictures which I took. A few of them were posted on Twitter ( now the platform is X) while the event was running. Pictures that I am there have been salvaged from other sources and I wholeheartedly thank those who took those. I don’t want to hurt the feelings of anyone or to poach these. If I knew the person directly I would have asked the permission. Still, if any copyright owners want to take these off, please DM me on LinkedIn as those photos were taken from a LinkedIn post.

The event at St. Joseph’s Group of Institutions, Chennai was stupendous and a grand function. Take this opportunity to thank the staff and management as well as congratulate all those who took the extra effort to make it such a great one.

About 15 new security controls added to AWS Security Hub

AWS Security Hub announced the addition of 15 new security controls through their post yesterday. This should increase the number of controls available to 307. AWS services such as Amazon FSx and AWS Private Certificate Authority (AWS Private CA) are some of the newly added controls that were in demand also. More and enhanced controls of previously supported services like Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Kubernetes Service (Amazon EKS), and Amazon Simple Storage Service (Amazon S3) are also added with this release. For the full list of recently released controls and the AWS Regions in which they are available, suggested to review the Security Hub user guide from time to time.

To use the new controls, turn on the standard they belong to. Security Hub will then start evaluating your security posture and monitoring your resources for the relevant security controls. You can use central configuration to do so across all your organization accounts and linked Regions with a single action. If you are already using the relevant standards and have Security Hub configured to automatically enable new controls, these new controls will run without taking any additional action.

The original announcement on their site is here.

Reference Architecture

Reference architecture for a generic interface for Cloud Search on AWS with a broker in any lambda-supported runtime. For the particular implementation, I chose and used Node.js. Hence any client request is authorized from an API key and hits the aws api gateway which in turn invokes the lambda function. In this function internally the code will do necessary normalization and pass it on to aws Cloud Search and if any response the same is reformatted for adapting as aws api gateway response. Along with this functionality, the lambda broker will write a human-readable version of the request as analyzed from the request with request method as verb keywords and sort direction with a prefix of JSON property names, etc into AWS cloud watch with simple console.log methods. Tried to make it as generic as possible.

An event bridge scheduler will trigger another lambda which will analyze these human readable messages and try to detect any missing indexes which will be auto-created into the Cloud Search and updated into a config file on aws S3. Lots of production testing and fine tuning is pending along with necessary documentation as well as the AWS sam template to deploy the same. As of now, this is just a blueprint and the components are lying in different locations and need orchestration there are no plans to open this on any public repository. But anyone who wants to adopt the design is free to pick this and do it on his own without any commitment to me. By creating this with the self-learning capabilities this system can be used literally by many applications even those that already depend on some kind of custom clumsy backend.

A few real-time use cases could be community member databases, hospital patient records, pet shops and many more. Generally, the request methods should work like POST create a new record, PUT updates a record, DELETE deletes ( or trash ) a referenced record, and GET fetch according with proper documentation the feature can be defined as the client software is designed and developed.

The reference architecture drawing is attached here and that is just my thoughts. Please share if you think this is good enough.