Following links but not indexing them

Post Reply
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Following links but not indexing them

Post by Thunderstone »





I am trying to index a lotus notes database but the problem occurs where
I want it to travel certain links which end in ?OpenView&Start=* (* will
be a number) but I don't want them actually indexed, and to continue
following the links given off the new page. If it is helpful, I want
pages that end in ?OpenDocument indexed.

Thanks in advance

James


User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Following links but not indexing them

Post by Thunderstone »




You'll have to let it walk the intermediate pages then delete them when
the walk is complete. See http://www.thunderstone.com/gw2man/node33.html .

You should use -noindex on the walk, peform the deletions, then use -index:
gw -noindex http://...
gw -s "delete ...
...
gw -index


Post Reply