Variable Spider Depth

[jasondwitt]

We have a list of URLs that we want to spider but we'd
like to vary the spider depth by URL. Our source data is
something like :

URL, Spider Depth
http://www.bob.com, 5
http://www.jim.com, 1
http://www.joe.com, 2

I'm thinking we can just sort and extrct the url
list into separate page depth files and then
write a distinct webinator spider script for
each input file specifying depth.

I was just wondering if the webinator product allows
us to do something like that with a little more ease?

thanks,
jason

[mark]

That's they way to do it. A given gw invocation will walk all supplied urls to the same depth. Make sure you -wipetodo between batches. Also note that you will not be able to use -rewalk since only the last depth is remembered.