Walking using a list of URLs

Post Reply
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Walking using a list of URLs

Post by Thunderstone »



Dear Sir, I created a file with a list of URLs (url.lst) that i wanted
gw to walk, but it walked only on the URL of the first line of that
file, leaving the remaining lines inexplicably ignored. All the URLs
started at the first position of each line. Any hints to unveil this
mystery?
Thanks in advance for your help.
João.


User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Walking using a list of URLs

Post by Thunderstone »




I assume you don't get any error messages.?
What option(s) are you using?
Can you supply a few example urls?
Try using -v4 to see what gw thinks about your urls.


User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Walking using a list of URLs

Post by Thunderstone »



Mark Willson wrote:

I´ve got the following messages after issuing the command gw -a -g -v4
"&url.lst":

175 Table hosts not found in data dictionary
115 No such table: hosts
000 SQLExecute() failed with -1
http://www.ltid.inpe.br/funcionario/lista.html
0: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0 Retrieving
1: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0
178 Trying to insert duplicate value
(www.ltid.inpe.br/funcionario/lista.html) i
n index
1: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0
1: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0 Not interested
1: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0 Not interested
Visited 1 pages total
Remember to run "gw -index" to update the index when you finish a batch


gw -a -g -v4 "&url.lst"


The contents of "url.lst" was:

http://www.ltid.inpe.br/funcionario/lista.html
http://www.lac.inpe.br/info/emails


As you can see above I´ve used this option.

I hope, this time, I´ve given you all the necessary details, othewise
let me know.
João.


User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Walking using a list of URLs

Post by Thunderstone »

Post Reply