Page 1 of 1

Re-crawling initial site

Posted: Tue Oct 13, 1998 10:44 am
by Thunderstone


Gee Whiz!

First I indexed my site with the simple gw URL command. It was
quick and VERY impressive.

But I didn't specify to crawl all my .asp pages. So I wiped the
database, and crawled again, this time using -fasp. But alas, still
no .asp pages were returned when searching.

My question is, how do I get these .asp pages to show up in a
search?

Please help!

Jerry



Re-crawling initial site

Posted: Tue Oct 13, 1998 12:50 pm
by Thunderstone



That should work unless the .asp pages are under directories excluded
with -x or robots.txt . What's your whole command line?

See what urls are in your database with something like:
gw -st "select Url from html"