I am trying to speed up the spidering process, and don't need links or references indexed.
Using this string
gw -d/database -noindex -a -R -r -O -fshtml -fasp -fcfm -fjsp -fxml -t7 -z5000 -v4 "&list"
still produces a report like this
137: TotLinks: 1595, Links: 9/ 5, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 4, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 3, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 2, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 1, Good: 0, New: 0 Disallowed protocol
137: TotLinks: 1595, Links: 9/ 0, Good: 0, New: 0
http://www.eng.uwaterloo.ca/other/gtarcisi
137: TotLinks: 1595, Links: 9/ 0, Good: 0, New: 0 Retrieving
138: TotLinks: 1605, Links: 10/ 10, Good: 0, New: 0 Chasing 10 refs
138: TotLinks: 1605, Links: 10/ 10, Good: 0, New: 0 Off site
-dnscache is enabled and -unique is turned on
Will -L streamline this?
Thanks
Using this string
gw -d/database -noindex -a -R -r -O -fshtml -fasp -fcfm -fjsp -fxml -t7 -z5000 -v4 "&list"
still produces a report like this
137: TotLinks: 1595, Links: 9/ 5, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 4, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 3, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 2, Good: 0, New: 0 Duplicate
137: TotLinks: 1595, Links: 9/ 1, Good: 0, New: 0 Disallowed protocol
137: TotLinks: 1595, Links: 9/ 0, Good: 0, New: 0
http://www.eng.uwaterloo.ca/other/gtarcisi
137: TotLinks: 1595, Links: 9/ 0, Good: 0, New: 0 Retrieving
138: TotLinks: 1605, Links: 10/ 10, Good: 0, New: 0 Chasing 10 refs
138: TotLinks: 1605, Links: 10/ 10, Good: 0, New: 0 Off site
-dnscache is enabled and -unique is turned on
Will -L streamline this?
Thanks