indexing directories

Post Reply
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

indexing directories

Post by Thunderstone »



I am specifying certain directories I want searched with a -j options file
and a url list file, called by

wgw -msearchfiles "&urllist"

Then when I rewalk the site, the directories show up in the database, but
none of the pages in those directories. What do you think is going wrong?

Thanks,
Rob Gwin

______________________________________________________
Get Your Private, Free Email at http://www.hotmail.com


User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

indexing directories

Post by Thunderstone »



You've probably walked some pages after your big walk without using the
full option set. Gw only remembers the last set of walk options.
-rewalk uses the last set of walk options and all of the URLs
previously specified. This will give you an idea of what options are
currently remembered that rewalk will use:

gw -st "select Name,String from options where Profile='lastrun'"

If you're going to use rewalk, make sure you always use the same set of
options for each sub-walk.

Turning up verbosity to 5 (-v5) will also give you more info about each
encountered link and what gw thinks of it.



Post Reply