Export Cloudwatch Logs to AWS S3 – Deploy using SAM

With due reference to the blog which helped me in the right direction, the Tensult blogs article Exporting of AWS CloudWatch logs to S3 using Automation, though at some points I have deviated from the original author’s suggestion.

Some points are blindly my preference and some other due to the suggested best practices. I do agree that starters, would be better off with setting IAM policies with ‘*’ in the resource field. But when you move things into production it is recommended to use least required permissions. Also, some critical policies were missing from the assume role policy. Another unnecessary activity was the checking of the existence of s3 bucket and attempt to create if not exists, at each repeated execution. Again for this purpose the lambda role needed create bucket permission. All these were over my head, and the outcome is this article.

Well if you need CloudWatch logs to be exported to S3 for whatever reason, this could save your time a lot, though this needs to be run in every different region where you need to deploy the stack. Please excuse me as the whole article expects to have aws-cli and sam-cli pre-installed.

Continue reading “Export Cloudwatch Logs to AWS S3 – Deploy using SAM”

Piwik Analytics Custom Variables Bug

After a long gap, I had the opportunity to dig into Piwik Analytics, the latest version (2.13.1), which has many new features from the last one we were using. During the time when implementing the same, I wanted to apply some custom variables, which showd the logged in user, the internal reference numbers and some other parameters. Whatever I did according to the documentations, the custom variables were not showing up.
Continue reading “Piwik Analytics Custom Variables Bug”

Javascript API Credentials – Just a port

There is not much to write than to attribute the logic up to a post on online code generator. Well since this was written in php and I needed the same in javascript, there was some cutting corners, and finally the script which is attached came up.

randomstring

 

The vars defined hold the two array of strings, and the function generates the key. Calling the function with genKey(16, access_salt) will generate a 16 char random string from the defined array access_salt.

Download

Script to redirect based on date or time or day of the week

Recently made a mistake by accepting a project before the initial milestone was deposited. Though I completed the project, the client did not pay, instead commented SCAM and deleted the project. So I am posting this over here hoping that the beggar will pick from here. Or this will be of use to some one.

This is not much of a magic or php expertise. The configuration area requires a default configuration with the tag ( index ) ‘0’ and then on the indexes have some meanings. A four digit will be treated as time. Eg ‘1200’ will take as 12 noon and any time after that unless another time is configured. But if you put ‘1170’ that will also be taken as 12 noon and continue. If put ‘Tuesday’ any day that falls on Tuesday will use that configuration. Again a blank configuration means not to show anything.
Continue reading “Script to redirect based on date or time or day of the week”

Setting session timestamp limited to script scope in PHP

Lets start with analyzing a use case. Well at Saturn we required to run a cron
job which was written in php, with a lot of use for date functions, every day
starting Jan 1 2001 to Dec 31 2012. I did not dig much into override_function
to change the behavior of the date function, instead a small class was written
to handle the ticks, but the loop was run using a history table in mysql.

Continue reading “Setting session timestamp limited to script scope in PHP”

Securing Web Applications – Best Practices

Finding your web site hacked and defaced one fine moring would be a nightmare for any web application developer. For securing from the same there are some recommendations. Through out the past years, we have summarized our strategy on the expectation that this would help others to build applications that are difficult to penetrate. Mostly the content of the article would be refering to PHP scripting language, and most others should not be any different.

Continue reading “Securing Web Applications – Best Practices”

bz2url – a very simple url compressor for wordpress

Its been over a year since we started bz2.in url compression for inhouse projects at Saturn and is being in use for Asianet India and this blog. Recently for a new project we absolutely needed an integrated url shortner, which finally gave shape to the initial version.
Continue reading “bz2url – a very simple url compressor for wordpress”

Class DumpIO – Inspired by Apache mod_dumpio but reluctant to restart webserver

Though there are far and wide systems for live debugging, and the sort, for forensic or load analysis, our php-extjs framework did not have anything pre planned, other than some query loggers, and background processing systems to log into db etc. While recently the Master MySQL server started showing variations in the cacti patterns (normal was about 4 to 20 in working window, but was steady between 35 and 40 in the tantrum period), we started to worry and could not identify the situation. Also restarting all application servers and clearing the session store would immediately drop the MySQL fsync graph to a standard pattern. This is the time when I looked for a input logger for Apache, and found about the dumpio, but needed the webserver to be restarted. Actually the time was ripe that the application was in a tantrum, and the MySQL graphs showing about 35 fsyncs average.

Revisiting Importance of event logging in server side scripting and other articles on the net, the out come was a class with a single static method. This was designed to pick and log any input. This was later moved as the lite version, and a full version capable of capturing the output also was built.
Continue reading “Class DumpIO – Inspired by Apache mod_dumpio but reluctant to restart webserver”

Linux CPU usage and montioring using shell memcache and jquery

Recently in a project where the application was deployed across multiple servers, the Client QA as well as Support Team wanted a better monitoring of all the servers in the production. It was too much to provide everybody with shell access and ask them to monitor using top. Well after a lot of digging through the wonderful search index of Google. And with insights from Paul Colby vide his article Calculating CPU Usage from /proc/stat, and various comments of Memcache usage through telnet along with the /dev/tcp socket connections it was just a matter of using some nifty shell processing before I could store each machine cpu values, loadavg, and running tasks as a json encoded string into memcache on one of the hosts with hostname as the key.
Continue reading “Linux CPU usage and montioring using shell memcache and jquery”