How to Index list of Urls in Url.txt file

melvint9
Posts: 19
Joined: Wed Oct 03, 2001 11:09 am

How to Index list of Urls in Url.txt file

Post by melvint9 »

How do you crawl a list of URLs in a Url.txt file a depth of 2. I tried:
gw -noindex -D1 http://localhost/test/url.txt
gw -D2 http://localhost/test/url.txt

I keep receiving, "Visited 1 pages total". It's not doing what I want, which is to index the urls listed in the URL.txt file.

Please help
User avatar
mark
Site Admin
Posts: 5519
Joined: Tue Apr 25, 2000 6:56 pm

How to Index list of Urls in Url.txt file

Post by mark »