Basic Applications:

Rpsoft 2000 Site Crawler is a helpful and powerful utility for webmasters!  For sites that are simply constructed for their links, this rpsoft 2000 site crawler can help with a number of common webmaster challenges, such as:  checking for reciprocal links, general site searches, email address finding, meta tag help, external link count and keyword density.

Reciprocal Links - a Handy Link Checker - a Super Time Saver

One of the biggest time savers of this program is looking for reciprocal links on partner sites back to you.  If you have done link exchanges, you will have noticed that not all sites tell you what page they have put your link on.  Even if they do tell you the web page, they may have to make changes later to put your link on another page - perhaps due to alphabetical order and the number of links that that site grows to.  And some sites have hundreds or even thousands of pages.  Site crawler can be sent to scan sites or multiple sites to look for your links - and will follow the internal links within those sites - if those links are constructed simply.  The maximum number of pages can be set in "options" but the overall maximum page scan per site is 5000 pages.

How important is it to check for partner reciprocal links?  We unfortunately find it very important.  We find that about a third of the sites that say that they will link to us immediately in fact never do.  And some of the ones that do may take as long as a month to get the link up back to our site.  Of the sites that do complete the link exchange that about 17% of them drop the link later.  A few of these sites go off of the air, and but even more of the 17% simply drop the links.  Adding these two numbers together, the number of sites we find problems in link exchange to be quite high.  So, unfortunately, we find it necessary to check partner links.  More on problems with link exchanges and why reciprocal links need to be verified by:  clicking here.

Basic Limitations - Important

This bot (short for software robot) uses a fast html scan technique rather than interpreting the code on the web page.  Therefore it will work for many sites (if simply constructed) but will not work on all sites.  The main issue here is on site scan or multi-site scan.  For site scan the sites must have their links to other pages done with simple html references (href=) and not complex javascript or mouseovers or other complexities.  Of course, one could argue that one should keep linkages simple, to ensure that search engine bots also can scan the sites and give the sites and their link partners proper credit for their popularity.  Site crawler also works best on site scans of main sites  rather than a subdirectory within a main site.

How does all of this impact results?  After many months of software refinement we tested site-crawler on our own rpsoft 2000 links.  We were happy to see that it was over 97% accurate.  By that we mean that it found the links or accurately reported them missing 97% of the time.  Of course your results could differ depending on your selection of sites, with simple or complex link structures.   In our case, our accuracy number is now going to be 100% using this software, since we are no longer initiating links to sites with complex link structures, and in following advice on the upcoming pages on problem sites that we now link to.

Some Site-Crawler Specifications:

• Operating System:  Windows systems 2000, or XP
• Search Types:  word search (can be used to find links also), email addresses
• Search Modes: web page, web site, or multiple web sites
• Other Primary Features:  Meta Tag Help, External Link Count (per page / per site) and Keyword Density (per page)
• Max Web Pages per site - can be set in "options", but 5000 is the absolute program maximum
• URLs Saved - up to 50 URLs can be saved after single entry of each (number 50 or less set in options)
• Startup URL - can be set to your preferred site name.  Also, the startup "search word" can be set to your link needs
• Operator Intervention - "Halt" provided for all modes, and "skip this site" available for multi site.  Both of these functions will be delayed slightly from the time of usage.

Detailed Features (assumes simple link construction on a site)

• Reciprocal Link Checker:  Can scan simple web pages or sites or multiple sites for reciprocal links.  Note - since html coders sometimes operate differently and may in fact add spaces, best to keep link search to a single word or collection of characters directly in a row such as www.mysite.com or simply mysite (no spaces). More on this here. and on web site promotion by clicking here.
• General Site Search:  Can scan simple web pages or sites or multiple sites for simple search words. Note that searching for reciprocal links is just a specific type of this general search routine.  More on this general application here.
• Search for next occurrence?  For single page or web site mode, one may search for the next occurrence of the search word also.  This of course is not done in "multi" or automatic mode when multiple sites are loaded.
• Email Finder: Can scan simple web pages or sites or multiple sites looking for email addresses.  Please use this only for legitimate contact needs and not for spam.
• Show Real HTML:  Can display actual html on a page.  Note that some browsers will redirect or pull in "src" documents and not simply show original and simple html for a page.
• Meta Tags:  Can show meta tags on a page for title, description and keywords and give an opinion as to their correct size.  It is possible to change the preferred size opinion within "options" if you wish a different standard than the one preset for you.  More on this application by clicking here.
• Sample Meta Tag:  Can show a sample meta tag area for those new at it
• Count External Links: Can show total external links on each of your web pages.  Some webmasters like to give a count of links in each category.  This is a big help for that.  Also there is a belief that going over 50 external links on a page will be ignored by some search engines - so this counting can help safeguard that also.
• Keyword Density:  Can review your keywords listed in a meta tag on a web page and tell you how many times each of those keyword phrases appear on that page (best if phrases separated by commas in keyword meta tag to facilitate this)
• Removing Links:  Are you deleting a web page?  Use the site crawler to scan your own site looking for links to that page you just eliminated to make sure no dead ends
• Multiple Modes:  For searches - general, link or email, it can search a single page, a web site, or multiple loaded sites
• Shortcut to Page:  If you know on which page of a partner site to look for your link, you can also load specific web page urls for multi-site usage or for checking a single web page.  If the shortcut fails and the link has moved, search the whole site as normal.
• Extensive options - that among other things allow you to enter your own default web site name for startup, or give priority searching certain pages over others or more
• Site Skip:  Option to skip a single site manually - during multi site operation - if it is going on longer than makes sense.  Sites that are skipped by the user are marked that way
• Stores Last Used URLs:  yes if the urls are typed in and options are saved (up to 50 URLs)
• Load and Save:  Load and Save available for loading multiple site lists for checking

more site crawler product pictures and usage

site-crawler test drive and results on rpsoft 2000 links

More on site-crawler Usage Adjustment, and Troubleshooting

reasons why checking for reciprocal links is a good idea

Other Limitations

While rpsoft 2000 site-crawler has a multi-site mode, jams can occur given certain internet problems in responses.  We suggest that when using multi-mode to check sites in smaller groups and save intermediate results so as to ensure less loss of data.

Loading Information

The rpsoft 2000 site crawler uses the latest Microsoft wininit.dll module.  You may need to update your windows 2000 computer (if it is not already updated) to the latest Microsoft Internet Explorer (free on the web from Microsoft) before Microsoft will allow the loading and usage of the more advanced wininit.dll module.  More on this on the rpsoft 2000 site crawler usage page.

Summary

Yes, some limitations, and one must keep their own site with simple links to take advantage of this fast html scan bot.  However, we find that this site crawler can save much much drudgery time on webmaster functions.  Even if it cannot scan all sites, any site that it gives a fast answer on - is one that you do not need to check manually.  Help file included with program

Links:  main page rpsoft 2000 software