spidering pages that don't end with a ".xxx"

Post Reply
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

spidering pages that don't end with a ".xxx"

Post by Thunderstone »



Hi,

I know how to use the -f switch to include non-standard pages.

However, I am trying to index an area of my site where the pages don't have
an extension at all. The url simply ends /bbs with any one of a number of
paramaters attached.

EG:

/bbs?ShowPost=145
/bbs?ShowPost=135
etc
etc


Is there a way to get webinator to recognize these as pages and then spider
through them?

david


==========================================================

David Cohen Jackie Needleman
david@parentsplace.com jackie@parentsplace.com

Dad and Mom to Noah

Please visit the Parenting Resource Center on the Web:

ParentsPlace.com
http://www.parentsplace.com/

==========================================================




User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

spidering pages that don't end with a ".xxx"

Post by Thunderstone »




You would have to use the -y option to fetch those.
This may be undesireable as it will also go after everything else too.

Webinator 2 will be able to fetch them without using -y.


Post Reply