Abandoning new walk

david180 · Post by **david180** » Wed Jun 08, 2005 2:12 pm

I am trying to walk a webserver that is protected by basic authentication. I have provided Webinator the credential, but I fail to walk as follows:

Walk started at 2005-06-08 12:00:46 (by resume)
JavaScript walking not enabled by current license
HTTPS walking disabled
Start fetching at http://zek.netdocuments.com/dcWeb/dcWeb.aspx
Ignore urls containing any of the following:
/cgi-bin/
~
?

started 1 new (1368) on http://zek.netdocuments.com/dcWeb/dcWeb.aspx
1 pages fetched (0 bytes) from http://zek.netdocuments.com/dcWeb/dcWeb.aspx
0 errors
0 duplicate pages

Updating search index ...Done.
Creating spell-checker dictionaries...Done.
Verifying usability of new walk.
Abandoning new walk. Cannot generate test query: No usable terms in index xh_TiDsKyMtBy_ViMoDpPp.

Post by **mark** » Wed Jun 08, 2005 4:47 pm

Look under List/edit urls to see what it found on that page. Click on the "Children" there to see what urls were found on that page.

Turn verbosity up to 4 and do a walk with mode set to new instead of refresh to get more details about what links were found.

Make sure you've added the desired extensions to the extensions list and that the exclusions list doesn't have anything that would cause desireable urls to be excluded. If your urls have query strings (? followed by data) that you want to index turn off strip queries.

david180 · Post by **david180** » Wed Jun 08, 2005 5:57 pm

When I have this meta tag in the very first page fetched this problem occurs.

<meta name="ROBOTS" content="noindex,follow"/>

When I removed the tag, the walk completed properly.

Post by **mark** » Thu Jun 09, 2005 12:33 pm

What version of scripts are you using?

david180 · Post by **david180** » Thu Jun 09, 2005 12:40 pm

Webinator 5.1.10
$Id: dowalk.src,v 2.310 2005/02/23 18:19:41 kai Exp $

Webinator 5.1.10
$Id: search4.src,v 2.173 2005/02/21 22:01:14 mark Exp $

<whine>There are too many copies of these scripts such that I have struggled to identify which copy I should be playing with to make my customizations.</whine>

Post by **mark** » Thu Jun 09, 2005 12:55 pm

See if the current version from the website fixes your problem.
http://www.thunderstone.com/texis/site/ ... ripts.html

The one in the INSTALLDIR/texis/scripts directory is the one being executed in a default install. If in doubt, remove the .vtx file then access the script. The directory where the .vtx reappears is where they are being run.