walking gw on existant database fails

Post Reply
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

walking gw on existant database fails

Post by Thunderstone »



when we add new URL to existing database, gw is walking through old refs =
like these:

Getting http://194.8.12.142/robots.txt...Not there...Ok.
Getting http://195.62.128.36/robots.txt...Not there...Ok.
...

and when any of previously walked servers are down, gw stops and hangs =
up without comments.

so it is inpossible to walk new URL until all previously walked web =
servers are operational.

is it our mistake or is it possible to remove referencies to walked =
URLs?

I tried to find were they reside, but without results :(

thank you,
edmunds zalite=20



User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

walking gw on existant database fails

Post by Thunderstone »




If the server is not there gw will continue after a TCP timeout.
To eliminate your issue you can delete the list of "known" Urls from
the options table. This will also disable the ability to use -rewalk.

gw -st "delete from options where Name='URL'"

Also make sure you don't have any Urls left over in todo from aborted
previous walks.

gw -st "delete from todo"


Post Reply