Show (beta):
Join Help
  Home » robots.spider(tm) » Details

robots.spider(tm)

  • Profile
  • Download
  • Reviews
  • Share

Homepage: robots.spider(tm) Homepage

robots.spider(tm) reads the robots.txt file, then spiders the site (depth is adjustable in the downloaded code), to check for references to entries in the robots.txt file. robots.spider(tm) can help you avoid disclosing the names of directories you don't want indexed, that aren't referenced in your site code. It is an open source (PHP) script, you can run it for free off the demo link, or download the code, adjust the depth, and use it to check your sites from a different server.


Related
View Folder Size Pro
Have you ever asked yourself: What size does occupy this folder? Probably you tried...
RankQuest Meta Tag Analyzer - Free SEO Tool
Meta Tags tell the Search Engine what your page is all about, they also play a crucial...
Directory Report
Directory printer Print a directory tree of an entire disk or a single directory...
Take Command
Take Command offers a new approach to working in Windows, bringing users the power...
AllNetic Working Time Tracker
Tracks how much time you spend on different projects and tasks. Thanks to precise...
eZee FrontDesk Hotel Management
eZee FrontDesk is a general purpose hotel and motel software which is can be used...
WebOnkey Ribbon Creator
Create amazing side ribbons for any website. The ribbons can be used to promote special...
RoSEE
RoSee converts dynamic web pages to HTML, thus allowing the search engines to index...
tkSpeller
tkSpeller is a tiny utility that sits in the corner of your screen and does nothing...
Paragon Drive Backup Personal
Drive Backup creates a backup image of entire hard disk, including operating system...
Most Popular
Quixhibit
Quixhibit is a Windows application that generates HTML image galleries for publication...
MAPILab Toolbox
A set of 18 "must have" Microsoft Outlook 2000/XP/2003/2007 add-ins for...
Market News Analyzer
The power tool to read and analyze news. Knowledge mining from news. Features: RSS...
countries list (as mySQL file) and flags
an sql file that uses as data the official names of countries in english as given...
Topsite Adder
Topsite Adder is a revolutionary new program that allows you to add your sites to...
Paessler URL Recorder
Paessler URL Recorder helps to find out the URLs and POSTDATA strings that a user...
Domain Dashboard
Domain Dashboard completely automates the most tedious webmaster tasks associated...
Perfect Macro Recorder
Easily automate recurring boring tasks using Perfect Macro Recorder. Use Perfect...
eCorral.exe
eCorral seamlessly and securely integrates your desktop with the Internet to give...
LinkCheck Pro- Reciprocal link checker
LinkCheck Pro is a program for web masters who exchange links with other web sites...
Product Information
Listing Views84
Date Added05.11.2009
Downloads26 this week - 26 Overall



© 2012 - Transit Network - All Rights Reserved - All Copyrights & Trademarks are property of their respective owners. Terms of Service | Privacy Policy