Page 1 of 1
Limited Results Of Walk
Posted: Sat Jun 19, 2010 10:10 pm
by lbneal
The target site has some 3500+ actual pages.
Webinator only indexes about 176 of them?
I have tried a variety of walk settings with no improvement in results.
I have tried New and Refresh both die at that count.
Limited Results Of Walk
Posted: Sun Jun 20, 2010 1:18 pm
by John
There are a couple of possible reasons for a crawl stopping early. The two main ones are either not finding the links, or the links are configured to be excluded, either by robots.txt or one of the other settings, for example the extension is not in the allow list.
Increasing the verbosity to 4 will give information on all the URLs, and why they aren't followed.
Limited Results Of Walk
Posted: Sun Jun 20, 2010 2:04 pm
by lbneal
Thanks! That seems to be a big help!
The number of hits is growing...
Limited Results Of Walk
Posted: Sun Jun 20, 2010 7:59 pm
by lbneal
However the the number increased to only 185...
Out of 3500+ pages I can see only 44 in the database...
Limited Results Of Walk
Posted: Sun Jun 20, 2010 8:38 pm
by lbneal
John -
I am using the Free version of Webinator.
My usage is NOT Politically Correct.
If you can help me with this issue it would be appreciated.
Otherwise no problem.
The package obviously has usability issues fixing such would enhance the value...
Limited Results Of Walk
Posted: Mon Jun 21, 2010 6:29 pm
by lbneal
Thanks to the help of John my site is now pretty much working.
I had a programming issue on a couple of the target sites.