Sitemap Generator in PHP (also web crawler, and search engine data builder)

Category: PHP - Miscellaneous

Ever wish you could just open up a program and have it generate an index of all links on your site? You need this program, you need the search engines to have this precious file called a sitemap.xml. It?s so important you have this file that all search engines check it when they crawl your site.How to install1. Download all the files and unzip them in a browsable directory2. Edit config.inc and change the example.com in the URL_seed and DOMAIN to whatever site you wish to crawl and generate data for.3. Browse to the directory you unzipped in, then wait. The program will do it?s run.It?s to be noted that if you close the window while it?s running it will interrupt the process, but here is the good news. Just reopen it, it will find where it left off and pickup again. This is because what this crawler really does.This actual program is a search engine crawler as well. It?s reading every file and parsing out the words and creating a matrix of words and frequency. Just like Google it will create keyword densities. If this was coupled with the search engine we created you would have a fully function search engine and crawler.If you are interested in the search engine data and portion, please contact us and we could certainly help you. We also plan in the future to release a full package of this. Date: 06 May, 2011

Add a Comment

all are required fields

     
What do you think of this resource?

Select Your Rate:

Votes:0