I'm trying to crawl www.drleonards.com, but it only crawls 1,900 URLs. Google says over 7,000 are indexed. I stripped queries when crawling and didn't haev any redirect errors. Any ideas on how to crawl more pages? Settings I might want to change?
Not crawling all URLs
Not crawling all URLs
Do you know of any specific pages that are not being indexed, or how many pages there should be? Do you want to index just www.drleonards.com, or all of the drleonards.com domain? It is possible that you have different URLs for the same thing that are being considered as duplicates, or you may have some pages only accessible with a query string, in which case you would want to remove ? from the exclusions, and not strip queries. Did you add .cfm to the extensions?
John Turnbull
Thunderstone Software
Thunderstone Software