Webinator stopping at depth of 1

mmongeau · Post by **mmongeau** » Fri Dec 05, 2003 11:37 am

I just purchased Professional Webinator 4.4.3-Windows-w/plugin. I'm trying to index an internal test site that has over 1000 pages but Webinator is not going below a depth of 1.

I created a profile using the default settings. Depth is set to -1. Exclusions are only /cgi-bin/ and /scripts/. Pages that contain deeper links are being indexed but they are not being crawled.

Looking at List/Edit URLs it only shows links at depths of 0 and 1. Nothing else. I ran a walk with Verbose set to 4 and it showed that it ignored offsite links but no mention of links deeper than level 1.

What might the problem be?

Post by **mark** » Fri Dec 05, 2003 11:50 am

Not sure what you mean by "no mention". Run a walk with verbosity set to 4. In List/Edit URLs go to a page that contains links you expect to be walked but weren't. Click "Children". See if that link is listed and what error is listed next to it in parens.

Also check INSTALLDIR/texis/vortex.log to see if anything unusual happened during the walk.

mmongeau · Post by **mmongeau** » Fri Dec 05, 2003 12:53 pm

By "no mention" I meant the pages I expected to be walked did not even appear on the walk status page with an error message.

I rewalked with verbosity set to 4 with the same result. I looked at several pages that I know contain links to pages on the same site and under "Children" it just shows "-None-". It appears that only pages that are linked from the main entry page are being indexed.

There are no errors in vortex.log. The site also does not have a robots.txt that could be interfering.

Post by **mark** » Fri Dec 05, 2003 2:56 pm

The pages with no links aren't java only pages or something are they?

Maybe you want to open a ticket using the "Tech support" link on the left so you can provide the source of the pages in question for analysis.