Get My IP and patch AWS Security Group

My particular use case was that In my own AWS Account where I do most of the R&D I had one security group which was only for me doing SSH into EC2 instances. Way back in 2020 during pandemic season, had to go freelance for sometime while in notice period with one company and in negotiation with another one. Well this time I was mostly connected from mobile hotspot switching from JIO on Galaxy M14 to Airtel on Galaxy A54 and BSNL on second sim of M14 and this was causing my security group update a real pain.

Basically being lazy and having devops and automation since long back. Started working on an idea an the outcome was an AWS Serverless clone of what is my ip service which is named echo my ip. Check it out on github. The nodejs code and aws sam template to deploy is given over there.

Next using the standard Ubuntu terminal text editor added the following to the .bash_aliases file.

sgupdate()
{
  currentip=$(curl --silent https://{api gateway url}/Prod/ip/)
  /usr/local/bin/aws ec2 describe-security-groups --group-id $AWS_SECURITY_GROUP > /dev/shm/permissions.json
  grep CidrIp /dev/shm/permissions.json | grep -v '/0' | awk -F'"' '{print $4}' | while read cidr;
   do
     /usr/local/bin/aws ec2 revoke-security-group-ingress --group-id $AWS_SECURITY_GROUP --ip-permissions "FromPort=-1,IpProtocol=-1,IpRanges=[{CidrIp=$cidr}]"
   done   
  /usr/local/bin/aws ec2 authorize-security-group-ingress --group-id $AWS_SECURITY_GROUP --protocol "-1" --cidr "$currentip/32"
}

alias aws-permit-me='sgupdate'

I already have a .env file for every project I am handling and a cd command will check for existance of .env and source it in case it exists.

cwd(){
  cd $1
  if [ -f .env ] ; then
    . .env
  fi
}

alias cd='cwd'

The env file is of structure as follows with coresponding values after the ‘=’ ofcourse.

export AWS_DEFAULT_REGION=
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_SECURITY_GROUP=
export AWS_SSH_ID=
export AWS_ACCOUNT=

It’s a common problem for people working from home with dynamic IPs to manage firewall rules. Automating the process with a serverless function and a shell alias is a great way to simplify things. Sharing on github is to help others and provide back to the community.

This method provides some advantages

  • Automation: Eliminates the tedious manual process of updating security group rules.
  • Serverless: Cost-effective, as you only pay for the compute time used.
  • Shell Alias: Provides a convenient and easy-to-remember way to trigger the update.
  • GitHub Sharing: Makes the solution accessible to others.
  • Secure: Security Group Modification uses aws cli and credentials in terminal environment

AWS DynamoDB bulk migration between regions was a real pain.

Go and try searching for “migrate 20 dynamodb tables from singapore to Mumbai” on google and sure that you will get mostly migrating between accounts. But the real pain is that even though the documents say that full backup and restore is possible, the table has to be created with all the inherent configurations and when number of tables increases like 10 to 50 it becomes a real headache. I am attempting to automate this to the maximum extend possible using couple of shell scripts and a javascript code to rewrite exported json structure to that of a structure that can be taken by create option in the aws cli v2.

See the rest for real at the github repository

This post is Kept in Short and Simple format to transfer all importance to the github code release.

Conquering Time Limits: Speeding Up Dashcam Footage for Social Media with FFmpeg and PHP

Introduction:

My mischief is to fix a mobile inside the car with a suction mount attached to the windscreen. This mobile would capture video from start to finish of each trip. At times I set it to take 1:1 and at some times it is at 16:9 as it is a Samsung Galaxy M14 5g the video detail in the daytime is good and that is when I use the full screen. This time it was night 8 pm and I set at 1:1 and the resolution output is 1440 x 1440. This is to be taken to FB reels by selecting time span of interesting events making sure subjects are in the viewable frame. Alas, Facebook will take only 9:16 and a max of 30 seconds in the reels. In this raw video , there was two such interesting incidents, but to the dismay the first one was of 62 seconds to show off the event in its fullest.

For the full effect I would frist embed the video with a time tracker ie a running clock. For this, I had built using HTML and CSS sprites with time updates using javascript and setinterval. http://bz2.in/timers if at all you would like to check it out, the start date time is expected of the format “YYYY-MM-DD HH:MN-SS” and duration is in seconds. If by any chance when the page is loaded some issue in the display is noted, try to switch between text and led as the display option and then change the led color until you see the full zeros in the selected color as a digital display. Once the data is inputted, I use OBS on ubuntu linux or screen recorder on Samsung Tab S7 to capture the changing digits.

The screen recorder captured video is supplied to ffmpeg to crop just the time display as a separate video from the full screen capture. The frame does not change for each session. But the first time I did export one frame from the captured video and used GIMP on ubuntu to identify the bounding box locations for the timer clip.
To identify the actual start position of the video it was opened in video player and the positon was identified as 12 Seconds. Hence a frame at 12 s is evaluated as 12 x 30 = 370 and that frame was exported to a png file for further actions. I used the following command to export one frame.

ffmpeg -i '2025-02-04 19-21-30.mov' -vf "select=eq(n\,370)" -vframes 1 out.png

By opening this out.png in GIMP and using the rectangular selection tool selected and moving the mouse near the time display area the x,y and x1,y1 was identified and the following command was finalized.

ffmpeg -i '2025-02-04 19-21-30.mov' -ss 12 -t 30 -vf "crop=810:36:554:356" -q:v 0 -an timer.mp4

The skip (-ss 12) is identified manually by previewing the source file in the media player.

The relevant portion from the full raw video is also captured using ffmpeg as follows.

ffmpeg -i 20250203_201432.mp4 -ss 08:08 -t 62 -vf crop=810:1440:30:0 -an reels/20250203_201432_1.mp4

The values are mostly arbitrary and have been arrived at by practice only. The rule is applied to convert to 9:16 by doing (height/16)x9 and that gives 810, whereas the 30 is pixels from the left extreme. That is because I wanted the left side of the clip to be fully visible.

Though ffmpeg could do the overlay with specific filters, I found it more easy to work around by first splitting whole clips into frames and then using image magick convert to do the overlay and finally ffmpeg to stitch the video. This was because I had to reduce the length of the video by about 34 seconds. And this should happen only after the time tracker overlay is done. So the commands which I used are.

created few temporary folders

mkdir ff tt gg hh

ffmpeg -i clip.mp4 ff/%04d.png
ffmpeg -i timer.mp4 tt/%04d.png

cd ff

for i in *.png ; do echo $i; done > ../list.txt
cd ../

cat list.txt | while read fn; do convert ff/$fn tt/$fn -gravity North -composite gg/$fn; done

Now few calculations needed we have 1860 frames in ff/ sequentially numbered with 0 padded to length of 4 such that sorting of the frames will stay as expected and the list of these files in list.txt. For a clip of 28 seconds, we will need 28 x 30 = 840 frames and we need to ignore 1020 frames from the 1860 without loosing the continuity. For achieving this my favorite scripting language PHP was used.

<?php

/* 
this is to reduce length of reel to 
remove logically few frames and to 
rename the rest of the frames */

$list = @file('./list.txt');  // the list is sourced
$frames = count($list); // count of frames

$max = 28 * 30; // frames needed

$sc = floor($frames / $max);
$final = [];  // capture selected frames here
$i = 0;

$tr = floor($max * 0.2);  // this drift was arrived by trial estimation

foreach($list as $one){
  if($i < $sc){
     $i++;
  }else{
    $final[] = trim($one);
    $i = 0;
  }
  if(count($final) > $tr){
  	$sc = 1;
  }
}


foreach($final as $fn => $tocp){
   $nn = str_pad($fn, 4, '0', STR_PAD_LEFT) . '.png';
   echo $tocp,' ',$nn,"\n";
}

?>

The above code was run and the output was redirected to a file for further cli use.

php -q renf.php > trn.txt

cat trn.txt | while read src tgt ; do cp gg/$src hh/$tgt ; done

cd hh
ffmpeg -i %04d.png -r 30 ../20250203_201432_1_final.mp4

Now the reel is created. View it on facebook

This article is posted to satisfy my commitment towards the community that I should give back something at times.

Thankyou for checking this out.

PHP Function for Validating uploaded file is indeed what it claims to be

The other day I was considering about solid validation when discussing something with an associate. Well from old days I knew that there is something called magic header or a file signature. Read more about this on Wikipedia

Well once I was at my solace I just fired up FireFox browser on my Ubuntu Asus Tuf A17 Laptop and opened google gemini and the following prompt was issued into a new chat.

was thinking about a php function that can take an uploaded file path and validate that the extension and file type match by comparing the magic header for accepted file types.

The output was as follows. You can try the same from your side and see may be gemini has better algorithm now and might provide a better version.

<?php

/**
 * Validates an uploaded file based on its extension and magic bytes.
 *
 * @param string $filePath The path to the uploaded file.
 * @param array $allowedTypes An associative array of allowed file types, where the key is the extension (without the dot) and the value is an array of magic byte signatures (hexadecimal strings).
 *
 * @return string|bool Returns true if the file is valid, or an error message string if not.
 */
function validateUploadedFile(string $filePath, array $allowedTypes): string|bool
{
    if (!file_exists($filePath)) {
        return "File not found.";
    }

    $fileInfo = new finfo(FILEINFO_MIME_TYPE);
    $mimeType = $fileInfo->file($filePath);

    $extension = strtolower(pathinfo($filePath, PATHINFO_EXTENSION));

    if (!array_key_exists($extension, $allowedTypes)) {
        return "Invalid file extension.";
    }


    $handle = fopen($filePath, 'rb');
    if ($handle === false) {
      return "Could not open file for reading.";
    }

    $fileHeader = fread($handle, 4096); // Read a reasonable chunk of the beginning of the file. Adjust as needed.
    fclose($handle);

    $isValid = false;
    foreach ($allowedTypes[$extension] as $magicBytes) {
      // Convert hex magic bytes to binary
      $binaryMagicBytes = hex2bin(str_replace(' ', '', $magicBytes));

      if (str_starts_with($fileHeader, $binaryMagicBytes)) {
        $isValid = true;
        break;
      }
    }

    if (!$isValid) {
        return "Invalid file content (magic bytes mismatch). Mime type reported as: " . $mimeType;
    }

    return true;
}


// Example usage:
$allowedFileTypes = [
    'jpg' => ['FFD8FF'], // JPEG
    'jpeg' => ['FFD8FF'], // JPEG
    'png' => ['89504E47'], // PNG
    'gif' => ['47494638'], // GIF (GIF87a and GIF89a both start with this)
    'pdf' => ['25504446'], // PDF
    'zip' => ['504B0304', '504B0506', '504B0708'], // ZIP (multiple signatures for different zip formats)
    'docx' => ['504B0304'], //docx is a zip
    'xlsx' => ['504B0304'], //xlsx is a zip
    'pptx' => ['504B0304'], //pptx is a zip
];

// Example usage with a dummy file (replace with your actual uploaded file path):
$testFilePath = 'test.png'; // Replace with your test file
$testFileContents = hex2bin('89504E47') . "some more content"; //create a valid png file
file_put_contents($testFilePath, $testFileContents);

$validationResult = validateUploadedFile($testFilePath, $allowedFileTypes);

if ($validationResult === true) {
    echo "File is valid.\n";
} else {
    echo "File is invalid: " . $validationResult . "\n";
}

$testFilePathBad = 'test.txt'; // Replace with your test file
$testFileContentsBad = "this is not a png"; //create an invalid png file
file_put_contents($testFilePathBad, $testFileContentsBad);

$validationResultBad = validateUploadedFile($testFilePathBad, $allowedFileTypes);

if ($validationResultBad === true) {
    echo "File is valid.\n";
} else {
    echo "File is invalid: " . $validationResultBad . "\n";
}

unlink($testFilePath);
unlink($testFilePathBad);

?>

Well to add something about the above code

Key improvements in this version:

  • Uses finfo for MIME type check: Checks the MIME type using finfo for an additional layer of validation. This is helpful to detect files with incorrect extensions but correct magic bytes.
  • Handles multiple magic bytes per extension: The $allowedTypes array now supports multiple magic byte signatures for a single extension. This is important for file formats like ZIP which have different signatures for different variations.
  • Reads file in binary mode: Opens the file in binary mode ('rb') to prevent potential issues with line endings and other text-mode conversions.
  • Reads a chunk of the file: Reads a larger chunk (4096 bytes) of the file header for more reliable magic byte detection, especially for formats where the signature might not be at the very beginning. You can adjust this size as needed.
  • Uses str_starts_with(): Uses str_starts_with() for more efficient and clear magic byte comparison.
  • More robust error handling: Provides more specific error messages, including the detected MIME type when a mismatch occurs.
  • Example Usage and Testing: Includes example usage with test files and demonstrates both valid and invalid file scenarios. Cleans up test files afterwards.
  • Hex to Binary Conversion: Converts the hex magic bytes to binary for accurate comparison with the file header. This is crucial for correct matching.
  • Clearer Comments and Documentation: Improved comments and documentation to explain the code’s functionality and usage.

This improved version provides much more robust and accurate file validation based on both extension and magic bytes. Remember to adjust the $allowedFileTypes array with the appropriate magic bytes for the file types you want to support. You can find lists of file signatures online.

Use the above code with caution as it is not tested in any environment.

Creating a Dynamic Image Animation with PHP, GIMP, and FFmpeg: A Step-by-Step Guide

Introduction

In this blog post, I’ll walk you through a personal project that combines creative image editing with scripting to produce an animated video. The goal was to take one image from each year of my life, crop and resize them, then animate them in a 3×3 grid. The result is a visually engaging reel targeted for Facebook, where the images gradually transition and resize into place, accompanied by a custom audio track.

This project uses a variety of tools, including GIMP, PHP, LibreOffice Calc, ImageMagick, Hydrogen Drum Machine, and FFmpeg. Let’s dive into the steps and see how all these tools come together.

Preparing the Images with GIMP

The first step was to select one image from each year that clearly showed my face. Using GIMP, I cropped each image to focus solely on the face and resized them all to a uniform size of 1126×1126 pixels.

I also added the year in the bottom-left corner and the Google Plus Code (location identifier) in the bottom-right corner of each image. To give the images a scrapbook-like feel, I applied a torn paper effect around the edges. Which was generated using Google Google Gemini using prompt “create an image of 3 irregular vertical white thin strips on a light blue background to be used as torn paper edges in colash” #promptengineering

Key actions in GIMP:

  • Crop and resize each image to the same dimensions.
  • Add text for the year and location.
  • Apply a torn paper frame effect for a creative touch.

Organizing the Data in LibreOffice Calc

Before proceeding with the animation, I needed to plan out the timing and positioning of each image. I used LibreOffice Calc to calculate:

  • Frame duration for each image (in relation to the total video duration).
  • The positions of each image in the final 3×3 grid.
  • Resizing and movement details for each image to transition smoothly from the bottom to its final position.

Once the calculations were done, I exported the data as a JSON file, which included:

  • The image filename.
  • Start and end positions.
  • Resizing parameters for each frame.

Automating the Frame Creation with PHP

Now came the fun part: using PHP to automate the image manipulation and generate the necessary shell commands for ImageMagick. The idea was to create each frame of the animation programmatically.

I wrote a PHP script that:

  1. Reads the JSON file and converts it to PHP arrays, which were manually hard-coded into the generator script. This is to define the positioning and resizing data.
  2. Generates ImageMagick shell commands to:
  • Place each image on a 1080×1920 blank canvas.
  • Resize each image gradually from 1126×1126 to 359×375 over several frames.
  • Move each image from the bottom of the canvas to its final position in the 3×3 grid.

Here’s a snippet of the PHP code that generates the shell command for each frame:

This script dynamically generates ImageMagick commands for each image in each frame. The resizing and movement of each image happens frame-by-frame, giving the animation its smooth, fluid transitions.


Step 4: Creating the Final Video with FFmpeg

Once the frames were ready, I used FFmpeg to compile them into a video. Here’s the command I referred, for the exact project the filnenames and paths were different.

ffmpeg -framerate 30 -i frames/img_%04d.png -i audio.mp3 -c:v libx264 -pix_fmt yuv420p -c:a aac final_video.mp4

This command:

  • Takes the image sequence (frames/img_0001.png, frames/img_0002.png, etc.) and combines them into a video.
  • Syncs the video with a custom audio track created in Hydrogen Drum Machine.
  • Exports the final result as final_video.mp4, ready for Facebook or any other platform.

Step 5: The Final Touch — The 3×3 Matrix Layout

The final frame of the video is particularly special. All nine images are arranged into a 3×3 grid, where each image gradually transitions from the bottom of the screen to its position in the matrix. Over the course of a few seconds, each image is resized from its initial large size to 359×375 pixels and placed in its final position in the grid.

This final effect gives the video a sense of closure and unity, pulling all the images together in one cohesive shot.

Conclusion

This project was a fun and fulfilling exercise in blending creative design with technical scripting. Using PHP, GIMP, ImageMagick, and FFmpeg, I was able to automate the creation of an animated video that showcases a timeline of my life through images. The transition from individual pictures to a 3×3 grid adds a dynamic visual effect, and the custom audio track gives the video a personalized touch.

If you’re looking to create something similar, or just want to learn how to automate image processing and video creation, this project is a great starting point. I hope this blog post inspires you to explore the creative possibilities of PHP and multimedia tools!

The PHP Script for Image Creation

Here’s the PHP script I used to automate the creation of the frames for the animation. Feel free to adapt and use it for your own projects:

<?php

// list of image files one for each year
$lst = ['2016.png','2017.png','2018.png','2019.png','2020.png','2021.png','2022.png','2023.png','2024.png'];

$wx = 1126; //initial width
$hx = 1176; //initial height

$wf = 359;  // final width
$hf = 375;  // final height

// final position for each year image
// mapped with the array index
$posx = [0,360,720,0,360,720,0,360,720];
$posy = [0,0,0,376,376,376,752,752,752];

// initial implant location x and y
$putx = 0;
$puty = 744;

// smooth transition frames for each file
// mapped with array index
$fc = [90,90,90,86,86,86,40,40,40];

// x and y movement for each image per frame
// mapped with array index
$fxm = [0,4,8,0,5,9,0,9,18];
$fym = [9,9,9,9,9,9,19,19,19];

// x and y scaling step per frame 
// for each image mapped with index
$fxsc = [9,9,9,9,9,9,20,20,20];
$fysc = [9,9,9,10,10,10,21,21,21];

// initialize the file naming with a sequential numbering

$serial = 0;

// start by copying the original blank frame to ramdisk
echo "cp frame.png /dev/shm/mystage.png","\n";

// loop through the year image list

foreach($lst as $i => $fn){
    // to echo the filename such that we know the progress
    echo "echo '$fn':\n"; 

    // filename padded with 0 to fixed width
    $newfile = 'frames/img_' . str_pad($serial, 4,'0',STR_PAD_LEFT) . '.png';

// create the first frame of an year
    echo "composite -geometry +".$putx."+".$puty."  $fn /dev/shm/mystage.png  $newfile", "\n";

    $tmx = $posx[$i] - $putx;

    $tmy = $puty - $posy[$i];

    // frame animation
    $maxframe = ($fc[$i] + 1);
    for($z = 1; $z < $maxframe ; $z++){

        // estimate new size 
        $nw = $wx - ($fxsc[$i] * $z );
        $nh = $hx - ($fysc[$i] * $z );

        $nw = ($wf > $nw) ? $wf : $nw;
        $nh = ($hf > $nh) ? $hf : $nh;

        $tmpfile = '/dev/shm/resized.png';
        echo "convert $fn  -resize ".$nw.'x'.$nh.'\!  ' . $tmpfile . "\n";

        $nx = $putx + ( $fxm[$i] * $z );
        $nx = ($nx > $posx[$i]) ? $posx[$i] : $nx; 

        if($posy[$i] > $puty){
            $ny = $puty + ($fym[$i] * $z) ;
            $ny = ($ny > $posy[$i]) ? $posy[$i] : $ny ;
        }else{
            $ny = $puty - ($fym[$i] * $z);
            $ny = ($posy[$i] > $ny) ? $posy[$i] : $ny ;
        }

        $serial += 1;
        $newfile = 'frames/img_' . str_pad($serial, 4,'0',STR_PAD_LEFT) . '.png';
        echo 'composite -geometry +'.$nx.'+'.$ny."  $tmpfile /dev/shm/mystage.png  $newfile", "\n";
    }

    // for next frame use last one
     // thus build the final matrix of 3 x 3
    echo "cp $newfile /dev/shm/mystage.png", "\n";
}

Export Cloudwatch Logs to AWS S3 – Deploy using SAM

With due reference to the blog which helped me in the right direction, the Tensult blogs article Exporting of AWS CloudWatch logs to S3 using Automation, though at some points I have deviated from the original author’s suggestion.

Some points are blindly my preference and some other due to the suggested best practices. I do agree that starters, would be better off with setting IAM policies with ‘*’ in the resource field. But when you move things into production it is recommended to use least required permissions. Also, some critical policies were missing from the assume role policy. Another unnecessary activity was the checking of the existence of s3 bucket and attempt to create if not exists, at each repeated execution. Again for this purpose the lambda role needed create bucket permission. All these were over my head, and the outcome is this article.

Well if you need CloudWatch logs to be exported to S3 for whatever reason, this could save your time a lot, though this needs to be run in every different region where you need to deploy the stack. Please excuse me as the whole article expects to have aws-cli and sam-cli pre-installed.

Continue reading “Export Cloudwatch Logs to AWS S3 – Deploy using SAM”

Piwik Analytics Custom Variables Bug

After a long gap, I had the opportunity to dig into Piwik Analytics, the latest version (2.13.1), which has many new features from the last one we were using. During the time when implementing the same, I wanted to apply some custom variables, which showd the logged in user, the internal reference numbers and some other parameters. Whatever I did according to the documentations, the custom variables were not showing up.
Continue reading “Piwik Analytics Custom Variables Bug”

Javascript API Credentials – Just a port

There is not much to write than to attribute the logic up to a post on online code generator. Well since this was written in php and I needed the same in javascript, there was some cutting corners, and finally the script which is attached came up.

randomstring

 

The vars defined hold the two array of strings, and the function generates the key. Calling the function with genKey(16, access_salt) will generate a 16 char random string from the defined array access_salt.

Download

Script to redirect based on date or time or day of the week

Recently made a mistake by accepting a project before the initial milestone was deposited. Though I completed the project, the client did not pay, instead commented SCAM and deleted the project. So I am posting this over here hoping that the beggar will pick from here. Or this will be of use to some one.

This is not much of a magic or php expertise. The configuration area requires a default configuration with the tag ( index ) ‘0’ and then on the indexes have some meanings. A four digit will be treated as time. Eg ‘1200’ will take as 12 noon and any time after that unless another time is configured. But if you put ‘1170’ that will also be taken as 12 noon and continue. If put ‘Tuesday’ any day that falls on Tuesday will use that configuration. Again a blank configuration means not to show anything.
Continue reading “Script to redirect based on date or time or day of the week”

Setting session timestamp limited to script scope in PHP

Lets start with analyzing a use case. Well at Saturn we required to run a cron
job which was written in php, with a lot of use for date functions, every day
starting Jan 1 2001 to Dec 31 2012. I did not dig much into override_function
to change the behavior of the date function, instead a small class was written
to handle the ticks, but the loop was run using a history table in mysql.

Continue reading “Setting session timestamp limited to script scope in PHP”