Web Spider using php cli

Purely on academic basis, I had once helped to cook up a web spider, which is used to build site maps. The spider was written in php and uses a couple of reg-exp matches, and finally writes the full sitemap from the start url. The system is assembled using two classes, WebPage and WebSpider. Then to make it similar to linux utilities in the command line environment, some functions were scooped in.
Continue reading “Web Spider using php cli”