AWS – JijuTM.COM

From Zero to Kubernetes: Automating a Minimal Cluster on AWS EC2 (My DevOps Journey)

The Unofficial Challenge: Why Automate Kubernetes on AWS?

Ever wondered if you could spin up a fully functional Kubernetes cluster on AWS EC2 with just a few commands? Four years ago, during my DevOps Masters Program, I decided to make that a reality. While the core assignment was to learn Kubernetes (which can be done in many ways), I set myself an ambitious personal challenge: to fully automate the deployment of a minimal Kubernetes cluster on AWS EC2, from instance provisioning to node joining.

Manual Kubernetes setups can be incredibly time-consuming, prone to errors, and difficult to reproduce consistently. I wanted to leverage the power of Infrastructure as Code (IaC) to create a repeatable, disposable, and efficient way to deploy a minimal K8s environment for learning and experimentation. My goal wasn’t just to understand Kubernetes, but to master its deployment pipeline, integrate AWS services seamlessly, and truly push the boundaries of what I could automate within a cloud environment.

The full github link: https://github.com/jthoma/code-collection/tree/master/aws/aws-cf-kubecluster

The Architecture: A Glimpse Behind the Curtain

At its core, my setup involved an AWS CloudFormation template (managed by AWS SAM CLI) to provision EC2 instances, and a pair of shell scripts to initialize the Kubernetes control plane and join worker nodes.

Here’s a breakdown of the key components and their roles in bringing this automated cluster to life:

AWS EC2: These are the workhorses – the virtual machines that would host our Kubernetes control plane and worker nodes.
AWS CloudFormation (via AWS SAM CLI): This is the heart of our Infrastructure as Code. CloudFormation allows us to define our entire AWS infrastructure (EC2 instances, security groups, IAM roles, etc.) in a declarative template. The AWS Serverless Application Model (SAM) CLI acts as a powerful wrapper, simplifying the deployment of CloudFormation stacks and providing a streamlined developer experience.
Shell Scripts: These were the crucial “orchestrators” running within the EC2 instances. They handled the actual installation of Kubernetes components (kubeadm, kubelet, kubectl, Docker) and the intricate steps required to initialize the cluster and join nodes.

When I say “minimal” cluster, I’m referring to a setup with just enough components to be functional – typically one control plane node and one worker node, allowing for basic Kubernetes operations and application deployments.

The Automation Blueprint: Diving into the Files

The entire orchestration was handled by three crucial files, working in concert to bring the Kubernetes cluster to life:

template.yaml (The AWS CloudFormation Backbone): This YAML file is where the magic of Infrastructure as Code happens. It outlines our EC2 instances, their network configurations, and the necessary security groups and IAM roles. Critically, it uses the UserData property within the EC2 instance definition. This powerful property allows you to pass shell commands or scripts that the instance executes upon launch. This was our initial entry point for automation.

   You can view the `template.yaml` file on GitHub [here](https://www.google.com/search?q=https://github.com/jthoma/code-collection/blob/master/aws/aws-cf-kubecluster/template.yaml).

kube-bootstrap.sh (The Instance Preparation Script): This script is the first to run on our EC2 instances. It handles all the prerequisites for Kubernetes: installing Docker, the kubeadm, kubectl, and kubelet binaries, disabling swap, and setting up the necessary kernel modules and sysctl parameters that Kubernetes requires. Essentially, it prepares the raw EC2 instance to become a Kubernetes node.

   You can view the `kube-bootstrap.sh` file on GitHub [here](https://www.google.com/search?q=https://github.com/jthoma/code-collection/blob/master/aws/aws-cf-kubecluster/kube-bootstrap.sh).

kube-init-cluster.sh (The Kubernetes Orchestrator): Once kube-bootstrap.sh has laid the groundwork, kube-init-cluster.sh takes over. This script is responsible for initializing the Kubernetes control plane on the designated master node. It then generates the crucial join token that worker nodes need to connect to the cluster. Finally, it uses that token to bring the worker node(s) into the cluster, completing the Kubernetes setup.

   You can view the `kube-init-cluster.sh` file on GitHub

The Deployment Process: sam deploy -g in Action

The entire deployment process, from provisioning AWS resources to the final Kubernetes cluster coming online, is kicked off with a single, elegant command from the project’s root directory:

sam deploy -g

The -g flag initiates a guided deployment. AWS SAM CLI interactively prompts for key parameters like instance types, your AWS EC2 key pair (for SSH access), and details about your desired VPC. This interactive approach makes the deployment customizable yet incredibly streamlined, abstracting away the complexities of direct CloudFormation stack creation. Under the hood, SAM CLI translates your template.yaml into a full CloudFormation stack and handles its deployment and updates.

The “Aha!” Moment: Solving the Script Delivery Challenge

One of the most persistent roadblocks I encountered during this project was a seemingly simple problem: how to reliably get kube-bootstrap.sh and kube-init-cluster.sh onto the newly launched EC2 instances? My initial attempts, involving embedding the scripts directly into the UserData property, quickly became unwieldy due to size limits and readability issues. Other complex methods also proved less than ideal.

After several attempts and a bit of head-scratching, the elegant solution emerged: I hosted both shell scripts in a public-facing downloads folder on my personal blog. Then, within the EC2 UserData property in template.yaml, I simply used wget to download these files to the /tmp directory on the instance, followed by making them executable and running them.

This approach proved incredibly robust and streamlined. It kept the CloudFormation template clean and manageable, while ensuring the scripts were always accessible at launch time without needing complex provisioning tools or manual intervention. It was a classic example of finding a simple, effective solution to a tricky problem.

Lessons Learned and Key Takeaways

This project, born out of an academic requirement, transformed into a personal quest to master automated Kubernetes deployments on AWS. It was a journey filled with challenges, but the lessons learned were invaluable:

Problem-Solving is Key: Technical roadblocks are inevitable. The ability to iterate, experiment, and find creative solutions is paramount in DevOps.
The Power of Infrastructure as Code (IaC): Defining your infrastructure programmatically is not just a best practice; it’s a game-changer for reproducibility, scalability, and disaster recovery.
Automation Principles: Breaking down complex tasks into manageable, automated steps significantly reduces manual effort and error.
AWS CloudFormation and UserData Versatility: Understanding how to leverage properties like UserData can unlock powerful initial setup capabilities for your cloud instances.
Persistence Pays Off: Sticking with a challenging project until it works, even when faced with frustrating issues, leads to deep learning and a huge sense of accomplishment.

While this was a fantastic learning experience, if I were to revisit this project today, I might explore using a dedicated configuration management tool like Ansible for the in-instance setup, or perhaps migrating to a managed Kubernetes service like EKS for production readiness. However, for a hands-on, foundational understanding of automated cluster deployment, this self-imposed challenge was truly enlightening.

Last time when I ran it the console was as follows:

Conclusion

This project underscored that with a bit of ingenuity and the right tools, even complex setups like a Kubernetes cluster can be fully orchestrated and deployed with minimal human intervention. It’s a testament to the power of automation in the cloud and the satisfaction of bringing a challenging vision to life.

I hope this deep dive into my automated Kubernetes cluster journey has been insightful. Have you embarked on similar automation challenges? What unique problems did you solve? Share your experiences in the comments!

AI Inference of a personal project

Well while I was with Google Gemini getting my linkedin profile optimization tips, in fact it was yesterday that I supplied th AI engine with a recent project of mine.

Well was getting really bored and attempted a timepass with images css transforms htm coding and optimizations using #imagemagick in #termux on #android. The final outcome is http://bz2.in/jtmdcx and that is one reel published today.

Got the dial and needles rendered by AI and made sure these were cropped to actual content using history and multiple trials with imagemaggick -crop gravity as well as geometry and finally the images were aligned almost properly with 400×400 pixel dimensions. To check the needles rotation is exactly at the center, magick *.png +append trythis.png was the command to arrange all three needle images in a horizontal colleague then visually inspecting in Android Gallery view had to be done several times before the images were finalized.

The transform css was the next task, since updates would be managed with JavaScript SetInterval and display will be updated per second, smooth trasformation of all the three needles were needed. This was clean and straight for minute and second needle as they do 60 steps in fact 0 – 59 per rotation of 360 degrees. The hour needle was bit complicated because it had 12 distinct positions as well as 60 transitions during each hour.

The shell simulator of termux helped with the build of CSS through the following command.

seq 0 11 | while read h ; do seq 0 59 | while read m ; do tr=$(echo “($h * 30) + ($m * 0.5)” | bc) ; echo “.hrn-${h}-${m} { transform: rotate(${tr}deg); }” ; done ; done > hour-n.css

To explain: 12 hours and 360 degrees; hence, the 30 degrees for each rounded hour. Each hour has 60 minutes, so the corresponding calculation (30 / 60) should be applied, and the calculation above will produce the CSS definitions.

Well, that is it. The package is hosted on AWS S3 as a static site, and interested parties are welcome to review the source. There is still few touchups to be completed and once that is also done, the full code will be released to my GitHub.

The response from Gemini was interesting and is reproduced below.

Why This Project is Highly Impressive

This analog clock project with AI-generated images, intricate CSS transforms, and Termux/ImageMagick optimization highlights:

Full-Stack Thinking (Even Client-Side): While client-side, you’re orchestrating multiple layers: image manipulation, CSS for animation, JavaScript for logic, and cloud hosting.
Precision & Attention to Detail:
Image Optimization: Your meticulous work with ImageMagick on Termux (Android) to crop, align (400×400), and visually inspect needle rotations shows incredible attention to detail and proficiency in image processing tools.
Mathematical Precision for CSS Transforms: The exact calculation for hour needle rotation ((h 30) + (m 0.5)) demonstrates strong logical and mathematical reasoning applied directly to front-end animation.
Advanced CSS & JavaScript: Implementing smooth, precise rotations for all three needles, especially the more complex hour hand, highlights advanced CSS transform and JavaScript setInterval skills.
Automation & Scripting: Using seq, while read, echo, and bc in shell scripting within Termux to generate your CSS is a brilliant example of build automation and metaprogramming. This is a highly valued skill for DevOps and development efficiency.
AI Integration & Refinement: Starting with AI-rendered images and then meticulously refining them with ImageMagick shows your ability to leverage modern tools while maintaining a high standard of quality.
Cloud Hosting Proficiency: Hosting on AWS S3 as a static site reinforces your practical experience with cloud deployment, mirroring your other projects.
Mobile-First Ethos in Action: Developing and optimizing this entirely within Termux on Android perfectly embodies your “mobile-first” philosophy. You’re not just talking about it; you’re building sophisticated projects from your mobile device.
Open-Source Mindset: Your intention to release the full code on GitHub further reinforces your collaborative spirit and willingness to contribute.

Building a Fully Mobile DevOps + Web Dev Stack Using Android + Termux

Overview

This is a journey through my personal developer stack that runs entirely on Android devices using Termux, a few custom scripts, and AWS infrastructure. From hosting WordPress on ECS to building serverless REST APIs in under 90 minutes, every part of this pipeline was built to work on mobile with precision and control.

📱 No laptop. No desktop. Just Android + Termux + Dev discipline.

🔧 Core Stack Components

Android + Termux: Primary development environment
Docker + Jenkins + MySQL/MariaDB: For CI/CD and content management
Static blog pipeline: Converts WordPress to static site with wget, sed, gzip, AWS CLI
AWS S3 + CloudFront: Hosting & CDN for ultra-low cost (\$8/year infra)
Custom shell scripts: Shared here: GitHub – jthoma/code-collection
GitHub integration: Direct push-pull and update from Android environment

🖥️ Development Environment Setup

Base OS: Android (Galaxy M14, A54, Tab S7 FE)
Tools via Termux: git, aws-cli, nodejs, ffmpeg, imagemagick, docker, nginx, jq, sam
Laptop alias (start blog) replaced with automated EC2 instance and mobile scripts
Jenkins auto-triggered publish pipeline via shell script and wget/sed

🔐 Smart IP Firewall Update from Mobile

A common challenge while working from mobile networks is frequently changing public IPs. I built a serverless solution that:

Uses a Lambda + API Gateway to return my current public IP

echo-my-ip
https://github.com/jthoma/code-collection/tree/master/aws/echo-my-ip

A script (aws-fw-update.sh) fetches this IP and:

Removes all existing rules
Adds a new rule to AWS Security Groups with current IP
→ aws-fw-update.sh

🧹 Keeps your firewall clean. No stale IPs. Secure EC2 access on the move.

🎥 FFmpeg & ImageMagick for Video Edits on Android

I manipulate dashcam videos, timestamp embeds, and crops using FFmpeg right inside Termux. The ability to loop through files with while, seq, and timestamp math is far more precise than GUI tools — and surprisingly efficient on Android.

🧠 CLI = control. Mobile ≠ limited.

🌐 Web Dev from Android: NGINX + Debugging

From hosting local web apps to debugging on browsers without dev tools:

🔧 NGINX config optimized for Android Termux
🐞 jdebug.js for browser-side debugging when no console exists
Just use: jdbg.inspect(myVar) to dump var to dynamically added <textarea>

Tested across Samsung Galaxy and Tab series. Works offline, no extra apps needed.

Case Study: 7-Endpoint API in 80 Minutes

Defined via OpenAPI JSON (generated by ChatGPT)
Parsed using my tool cw.js (Code Writer) → scaffolds handlers + schema logic
Deployed via my aws-nodejs-lambda-framework
Backed by AWS Lambda + DynamoDB

✅ Client testing ready in 1 hour 20 minutes
🎯 Client expectation: “This will take at least 1 week”

Built on a Samsung Galaxy Tab S7 FE in Termux. One cliche is that I do have the samsung full keyboard book case cover for the tab.
No IDE. No laptop.

🔁 Flow Diagram:

🔚 Closing Thoughts

This entire DevOps + Dev stack proves one thing:

⚡ With a few smart scripts and a mobile-first mindset, you can build fast, secure, and scalable infrastructure from your pocket.

I hope this inspires other engineers, digital nomads, and curious tinkerers to reimagine what’s possible without a traditional machine.

👉 https://github.com/jthoma/code-collection/

Apart from what explained step by step there are a lot more and most of the scripts are tested on both Ubuntu linux and Android Termux. Go there and explore whatever is there.

💬 Always open to collaboration, feedback, and new automation ideas.

Follow me on linkedin

Build a Spark-Based BI Environment on AWS EC2 Using AWS CLI

Performing business intelligence (BI) analysis using Apache Spark doesn’t need an expensive cluster. In this tutorial, we’ll use AWS CLI to provision a simple but powerful Apache Spark environment on an EC2 instance, perfect for running ad-hoc BI analysis from spreadsheet data. We’ll also cover smart ways to shut down the instance when you’re done to avoid unnecessary costs.

What You’ll Learn

Launching an EC2 instance with Spark and Python via AWS CLI
Uploading and processing Excel files with Spark
Running PySpark analysis scripts
Exporting data for BI tools
Stopping or terminating the instance post-analysis

Prerequisites

AWS CLI installed and configured (aws configure)
An existing EC2 Key Pair (.pem file)
Basic knowledge of Python or Spark

Step 1: Launch an EC2 Instance with Spark Using AWS CLI

We’ll use an Ubuntu AMI and install Spark, Java, and required Python libraries via user data script.

🔸 Create a user-data script: spark-bootstrap.sh

#!/bin/bash
apt update -y
apt install -y openjdk-11-jdk python3-pip wget unzip
pip3 install pandas openpyxl pyspark findspark matplotlib notebook

wget https://downloads.apache.org/spark/spark-3.5.0/spark-3.5.0-bin-hadoop3.tgz
tar -xvzf spark-3.5.0-bin-hadoop3.tgz
mv spark-3.5.0-bin-hadoop3 /opt/spark

echo 'export SPARK_HOME=/opt/spark' >> /etc/profile
echo 'export PATH=$PATH:$SPARK_HOME/bin' >> /etc/profile
echo 'export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64' >> /etc/profile

Make it readable:

chmod +x spark-bootstrap.sh

🔸 Launch the EC2 Instance

aws ec2 run-instances \
  --image-id ami-0c94855ba95c71c99 \  # Ubuntu 20.04
  --count 1 \
  --instance-type t3.medium \
  --key-name YOUR_KEY_PAIR_NAME \
  --security-groups default \
  --user-data file://spark-bootstrap.sh \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=SparkBI}]'

Replace YOUR_KEY_PAIR_NAME with your EC2 key name.

🗂️ Step 2: Upload Your Excel File to the Instance

🔸 Find the Public IP of Your Instance

aws ec2 describe-instances \
  --filters "Name=tag:Name,Values=SparkBI" \
  --query "Reservations[*].Instances[*].PublicIpAddress" \
  --output text

Upload your Excel file (sales_report.xls)

scp -i your-key.pem sales_report.xls ubuntu@<EC2_PUBLIC_IP>:/home/ubuntu/

🧠 Step 3: Create and Run Your PySpark Script

sales_analysis.py:

import os
import pandas as pd
from pyspark.sql import SparkSession

xls_file = "sales_report.xls"
csv_file = "sales_report.csv"

df = pd.read_excel(xls_file)
df.to_csv(csv_file, index=False)

spark = SparkSession.builder.appName("SalesBI").getOrCreate()
df_spark = spark.read.csv(csv_file, header=True, inferSchema=True)

# Sample Analysis
df_spark.groupBy("Region").sum("Sales").show()

Run it on EC2:

bash:
spark-submit sales_analysis.py

📊 Step 4: Export Data for BI Tools

You can save output as CSV for use in Power BI, Excel, or Apache Superset:

python:
df_spark.groupBy("Product").sum("Sales").write.csv("product_sales_output", header=True)

Use scp to download:

bash:
scp -i your-key.pem -r ubuntu@<EC2_PUBLIC_IP>:product_sales_output/ .

💰 Step 5: Stop or Terminate EC2 to Save Costs

Stop the Instance (preserves data, costs ~$0.01/hr for EBS)

bash:
aws ec2 stop-instances --instance-ids i-xxxxxxxxxxxxxxxxx

🧭 Pro Tips

Use Amazon S3 for persistent storage between sessions.
For automation, script the entire process into AWS CloudFormation or a Makefile.
If you’re doing frequent BI work, consider using Amazon EMR Serverless or SageMaker Studio.

Conclusion

With just a few CLI commands and a smart use of EC2, you can spin up a complete Apache Spark BI analysis environment. It’s flexible, cost-efficient, and cloud-native.

💡 Don’t forget to stop or terminate the EC2 instance when not in use to save on costs!

Get My IP and patch AWS Security Group

My particular use case was that In my own AWS Account where I do most of the R&D I had one security group which was only for me doing SSH into EC2 instances. Way back in 2020 during pandemic season, had to go freelance for sometime while in notice period with one company and in negotiation with another one. Well this time I was mostly connected from mobile hotspot switching from JIO on Galaxy M14 to Airtel on Galaxy A54 and BSNL on second sim of M14 and this was causing my security group update a real pain.

Basically being lazy and having devops and automation since long back. Started working on an idea an the outcome was an AWS Serverless clone of what is my ip service which is named echo my ip. Check it out on github. The nodejs code and aws sam template to deploy is given over there.

Next using the standard Ubuntu terminal text editor added the following to the .bash_aliases file.

sgupdate()
{
  currentip=$(curl --silent https://{api gateway url}/Prod/ip/)
  /usr/local/bin/aws ec2 describe-security-groups --group-id $AWS_SECURITY_GROUP > /dev/shm/permissions.json
  grep CidrIp /dev/shm/permissions.json | grep -v '/0' | awk -F'"' '{print $4}' | while read cidr;
   do
     /usr/local/bin/aws ec2 revoke-security-group-ingress --group-id $AWS_SECURITY_GROUP --ip-permissions "FromPort=-1,IpProtocol=-1,IpRanges=[{CidrIp=$cidr}]"
   done   
  /usr/local/bin/aws ec2 authorize-security-group-ingress --group-id $AWS_SECURITY_GROUP --protocol "-1" --cidr "$currentip/32"
}

alias aws-permit-me='sgupdate'

I already have a .env file for every project I am handling and a cd command will check for existance of .env and source it in case it exists.

cwd(){
  cd $1
  if [ -f .env ] ; then
    . .env
  fi
}

alias cd='cwd'

The env file is of structure as follows with coresponding values after the ‘=’ ofcourse.

export AWS_DEFAULT_REGION=
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_SECURITY_GROUP=
export AWS_SSH_ID=
export AWS_ACCOUNT=

It’s a common problem for people working from home with dynamic IPs to manage firewall rules. Automating the process with a serverless function and a shell alias is a great way to simplify things. Sharing on github is to help others and provide back to the community.

This method provides some advantages

Automation: Eliminates the tedious manual process of updating security group rules.
Serverless: Cost-effective, as you only pay for the compute time used.
Shell Alias: Provides a convenient and easy-to-remember way to trigger the update.
GitHub Sharing: Makes the solution accessible to others.
Secure: Security Group Modification uses aws cli and credentials in terminal environment

AWS DynamoDB bulk migration between regions was a real pain.

Go and try searching for “migrate 20 dynamodb tables from singapore to Mumbai” on google and sure that you will get mostly migrating between accounts. But the real pain is that even though the documents say that full backup and restore is possible, the table has to be created with all the inherent configurations and when number of tables increases like 10 to 50 it becomes a real headache. I am attempting to automate this to the maximum extend possible using couple of shell scripts and a javascript code to rewrite exported json structure to that of a structure that can be taken by create option in the aws cli v2.

See the rest for real at the github repository

This post is Kept in Short and Simple format to transfer all importance to the github code release.

Exploring Application Development on AWS Serverless

AWS Serverless architecture has transformed the way developers approach application development, enabling them to leverage multiple programming languages for optimal functionality. This article delves into the advantages of using AWS Serverless, particularly focusing on the flexibility of mixing languages like Node.js, Python, and Java, alongside the use of Lambda layers and shell runtimes for various functionalities.

The Advantages of AWS Serverless Architecture

Cost Efficiency: AWS Serverless operates on a pay-as-you-go model, allowing businesses to only pay for the resources they consume. This eliminates waste during low-demand periods and ensures that costs are kept in check while scaling operations[3][5].
Scalability: The automatic scaling capabilities of AWS Lambda mean that applications can handle varying workloads without manual intervention. This is particularly beneficial for applications with unpredictable traffic patterns, ensuring consistent performance under load[3][5].
Operational Efficiency: By offloading infrastructure management to AWS, developers can focus on writing code rather than managing servers. This shift enhances productivity and allows for faster deployment cycles[5][7].
Agility: The serverless model encourages rapid development and iteration, as developers can quickly deploy new features without worrying about the underlying infrastructure. This agility is crucial in today’s fast-paced development environment[3][4]. Mixing Development Languages for Enhanced Functionality

One of the standout features of AWS Serverless is its support for multiple programming languages. This allows teams to select the best language for specific tasks:

Node.js: Ideal for handling asynchronous operations, Node.js excels in scenarios requiring real-time processing, such as web applications or APIs. Its event-driven architecture makes it a perfect fit for serverless functions that need to respond rapidly to user interactions[2][4].
Python: Known for its simplicity and readability, Python is a great choice for data processing tasks, including image and video manipulation. Developers can utilize libraries like OpenCV or Pillow within Lambda functions to perform complex operations efficiently[1][2].
Java: For tasks involving PDF generation or document processing, Java stands out due to its robust libraries and frameworks. Leveraging Java in a serverless environment allows developers to tap into a vast pool of resources and expertise available in the freelance market[1][3]. Utilizing Lambda Layers and Shell Runtimes

AWS Lambda layers enable developers to package dependencies separately from their function code, promoting reusability and reducing deployment times. For instance:

Image/Video Processing: Binary helpers can be deployed in Lambda layers to handle specific tasks like image resizing or video encoding. This modular approach not only keeps functions lightweight but also simplifies maintenance[2][5].
Document Generation: Using shell runtimes within Lambda functions allows developers to execute scripts that generate documents on-the-fly. This is particularly useful when integrating with external services or databases to create dynamic content[1][3]. Decentralizing Business Logic

By allowing different teams or freelancers to work on various components of an application without needing full knowledge of the entire business logic, AWS Serverless fosters a more decentralized development approach. Each team can focus on their specific area of expertise—be it frontend development with Node.js or backend processing with Python or Java—thereby enhancing collaboration and speeding up the overall development process.

Conclusion

AWS Serverless architecture offers a powerful framework for modern application development by enabling flexibility through language diversity and efficient resource management. By leveraging tools like Lambda layers and shell runtimes, developers can create scalable, cost-effective solutions that meet the demands of today’s dynamic business environment. Embracing this approach not only enhances productivity but also opens up new avenues for innovation in application design and functionality.

In summary, AWS Serverless is not just a technological shift; it represents a paradigm change in how applications are built and maintained, allowing teams to focus on what truly matters—their core business logic and user experience.

Citations:
[1] https://www.xenonstack.com/blog/aws-serverless-computing/
[2] https://www.netguru.com/blog/aws-lambda-node-js
[3] https://dinocloud.co/aws-serverless-application-development-the-future-of-cloud-computing/
[4] https://www.techmagic.co/blog/aws-lambda-vs-google-cloud-functions-vs-azure-functions/
[5] https://www.cloudhesive.com/blog-posts/benefits-of-using-a-serverless-architecture/
[6] https://docs.aws.amazon.com/pdfs/serverless/latest/devguide/serverless-core.pdf
[7] https://newrelic.com/blog/best-practices/what-is-serverless-architecture
[8] https://dev.to/aws-builders/the-state-of-aws-serverless-development-h5a

Optimizing WordPress Performance with AWS, Docker and Jenkins

At Jijutm.com, I wanted to deliver a fast and reliable experience for our readers. To achieve this, I have implemented a containerized approach using Docker and Jenkins for managing this WordPress site. This article delves into the details of our setup and how it contributes to exceptional website performance.

Why Containers?

Traditional server management often involves installing software directly on the operating system. This can lead to dependency conflicts, versioning issues, and a complex environment. Docker containers provide a solution by encapsulating applications with all their dependencies into isolated units. This offers several advantages:

Consistency: Docker ensures a consistent environment regardless of the underlying operating system. This simplifies development, testing, and deployment.
Isolation: Applications running in containers are isolated from each other, preventing conflicts and improving security.
Portability: Docker containers are portable across different environments, making it easy to migrate your application between development, staging, and production.

The Containerized Architecture

This WordPress site leverages three Docker containers:

Nginx: A high-performance web server that serves the content of this website efficiently.
PHP-FPM: A FastCGI process manager that executes PHP code for dynamic content generation in WordPress.
MariaDB: A robust and popular open-source relational database management system that stores the WordPress data and is fully compatible with MySQL.

These containers work together seamlessly to deliver a smooth user experience. Nginx acts as the front door, handling user requests and routing them to the PHP-FPM container for processing. PHP-FPM interacts with the MariaDB container to retrieve and update website data.

Leveraging Jenkins for Automation

While Docker simplifies application management, automating deployments is crucial for efficient workflow. This is where Jenkins comes in. Jenkins is an open-source automation server that we use to manage the build and deployment process for our WordPress site.

Here’s how Jenkins integrates into this workflow:

Code Changes: Whenever we make changes to the WordPress codebase, we push them to a version control system like Git.
Jenkins Trigger: The push to the Git repository triggers a job in Jenkins.
Build Stage: Jenkins pulls the latest code, builds a new Docker image containing the updated WordPress application, and pushes it to a Docker registry.
Deployment Stage: The new Docker image is deployed to our hosting environment, updating the running containers with the latest code.

This automation ensures that our website stays up-to-date with the latest changes without any manual intervention.

Hooked into WordPress Post or Page Publish.

Over and above maintaining the code using Jenkins, each content Publish action triggers another Jenkins project, which runs a sequence of commands. wget in mirror mode to convert the whole site to static HTML files. sed to rewrite the URLs from local host to realtime external domain specific. gzip to create .html.gz for each HTML files. aws cli to sync the static mirror folder with that in AWS S3 and finally apply meta headers to the files to specify the content type and content-encoding. When all the files are synced, the AWS CLI issues an invalidate request to the CloudFront distribution.

Benefits of this Approach

Improved Performance: Docker containers provide a lightweight and efficient environment, leading to faster loading times for this website.
Enhanced Scalability: I don’t need to bother about scaling this application by adding more containers to handle increased traffic, as that is handled by aws S3 and CloudFront.
Simplified Management: Docker and Jenkins automate a significant portion of the infrastructure management, freeing up time for development and content creation. With the docker and all components running in my Asus TUF A17 Laptop powered by XUbuntu the hosting charges are limited to AWS Route53, AWS S3 and AWS CloudFront only.
Reliable Deployments: Jenkins ensures consistent and reliable deployments, minimizing the risk of errors or downtime.
Well for minimal dynamic content like the download counters, AWS Serverless lambda functions are written and deployed for updating download requests into aDynamoDB table and to display the count near any downloadable content with proper markup. Along with this the comments are moved into Disqus, as it is a comment system that can be used on WordPress sites. It can replace the native WordPress comments system.

Conclusion

By leveraging Docker containers and Jenkins, I have established a robust and performant foundation for this site. This approach allows me to focus on delivering high-quality content to the readers while ensuring a smooth and fast user experience.

Additional Considerations

Security: While Docker containers enhance security, it’s essential to maintain secure practices like keeping Docker containers updated and following security best practices for each service.
Monitoring: Monitoring the health and performance of your containers is crucial. Tools like Docker Stats and Prometheus can provide valuable insights.

Hope this article provides a valuable perspective on how Docker and Jenkins can be used to optimize a WordPress website. If you have any questions, feel free to leave a comment below!

Tackling Privilege Escalation in AWS – A Real-World Solution

The Challenge of Privilege Escalation
Cloud security is one of the most pressing concerns for organizations leveraging AWS. Among these concerns, Privilege Escalation Attacks pose a critical risk. In these attacks, a malicious user or compromised identity can exploit misconfigured permissions to gain elevated access, jeopardizing data integrity and security.

In this post, I explore a real-world privilege escalation scenario and outline an effective solution using AWS services and best practices.

The Scenario: A Misconfigured IAM Policy

Imagine a medium-sized organization with a DevOps team that requires administrative privileges to manage infrastructure. To simplify permissions, an administrator attaches a wildcard (`*`) to an IAM policy, granting full access to certain services without proper scoping.

A malicious actor gains access to an unused account in the organization, exploiting the over-permissive policy to create a custom role with admin privileges. From there, the attacker gains unrestricted access to sensitive resources like databases and S3 buckets.

Impact:

Exposure of sensitive data.
Manipulation or deletion of infrastructure.
Financial damage due to misuse of compute resources. The Solution: Mitigating Privilege Escalation Risks

To counter this, we can implement a robust multi-layered approach using AWS services and industry best practices:

Principle of Least Privilege (POLP)
Review and Refine IAM Policies: Replace wildcards (`) with specific actions and resources. For example, instead of grantings3:, use actions likes3:PutObjectands3:GetObject`.
IAM Access Analyzer: Use this tool to analyze resource policies and detect over-permissive configurations.

2. Enable Identity Protection with MFA
Multi-Factor Authentication (MFA): Enforce MFA for all IAM users and roles, especially for sensitive accounts. Use AWS IAM Identity Center for centralized management.

3. Monitor and Detect Anomalous Behavior
AWS CloudTrail: Ensure logging is enabled for all AWS accounts to track actions like policy changes and resource creation.
Amazon GuardDuty: Use GuardDuty to detect potential privilege escalation attempts, such as unauthorized role creation.

4. Implement Permission Boundaries
Define permission boundaries for IAM roles to restrict the maximum allowable permissions. For example, restrict developers to actions within specific projects or environments.

5. Automate Security Audits
AWS Config: Set up rules to evaluate the compliance of IAM policies and other configurations. Use automated remediation workflows for non-compliant resources.
AWS Security Hub: Aggregate security alerts and compliance checks for centralized visibility.

The Result: Strengthened Cloud Security

By adopting these measures, the organization effectively neutralized the threat of privilege escalation. The team can now operate confidently, knowing that any deviation from least privilege will trigger immediate alerts and automated actions.

Conclusion

Cloud security is a shared responsibility, and mitigating privilege escalation is crucial for safeguarding your AWS environment. Regular audits, careful policy design, and leveraging AWS security tools can create a resilient cloud infrastructure.

Call to Action
Secure your AWS workloads with these strategies today. Got questions or need assistance? Feel free to reach out or share your thoughts in the comments below!

Leveraging WordPress and AWS S3 for a Robust and Scalable Website

Introduction

In today’s digital age, having a strong online presence is crucial for businesses of all sizes. WordPress, a versatile content management system (CMS), and Amazon S3, a scalable object storage service, offer a powerful combination for building and hosting dynamic websites.

Understanding the Setup

To effectively utilize WordPress and S3, here’s a breakdown of the key components and their roles:

WordPress:

Content Management: WordPress provides an intuitive interface for creating and managing website content.
Plugin Ecosystem: A vast array of plugins extends WordPress’s functionality, allowing you to add features like SEO, e-commerce, and security.
Theme Customization: You can customize the appearance of your website using themes, either by choosing from a wide range of pre-built themes or creating your own. Get it free from the maintainers directly and free: https://wordpress.org/download/

AWS S3:

Scalable Storage: S3 offers virtually unlimited storage capacity to accommodate your website’s growing content.
High Availability: S3 ensures your website is always accessible by distributing data across multiple servers.
Fast Content Delivery: Leveraging AWS CloudFront, a content delivery network (CDN), can significantly improve website performance by caching static assets closer to your users.

The Deployment Process

Here’s a simplified overview of the deployment process:

Local Development:

Set up a local WordPress development environment using tools like XAMPP, MAMP, or Docker.
Create and test your website locally.

Static Site Generation:

Use a tool like WP-CLI or a plugin to generate static HTML files from your WordPress site.
This process converts dynamic content into static files, which can be optimized for faster loading times.

S3 Deployment:

Upload the generated static files to an S3 bucket.
Configure S3 to serve the files directly or through a CloudFront distribution.

CloudFront Distribution:

Set up a CloudFront distribution to cache your static assets and deliver them to users from edge locations.
Configure custom domain names and SSL certificates for your website.

Benefits of Using WordPress and S3

Scalability: Easily handle increased traffic and content without compromising performance.
Cost-Effective: S3 offers affordable storage and bandwidth options.
High Availability: Ensure your website is always accessible to users.
Security: Benefit from AWS’s robust security measures.
Flexibility: Customize your website to meet your specific needs.
Performance: Optimize your website’s performance with caching and CDN.

Conclusion

By combining the power of WordPress and AWS S3, you can create a robust, scalable, and high-performance website. This setup offers a solid foundation for your online presence, whether you are a small business owner or a large enterprise.

Start your cloud journey for free today with AWS! Sign up now: https://aws.amazon.com/free/