Don't walk IP address

Post Reply
mjacobson
Posts: 204
Joined: Fri Feb 08, 2002 3:35 pm

Don't walk IP address

Post by mjacobson »

I would like to exclude all IP address from being crawled. I am using gw and in the options file I added the following regexp x/>>=\digit{3}\. Is there a flag or some other way to keep webinator from walking any links that are IP address based?
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Don't walk IP address

Post by mark »

There's no option or such designed for doing that. -x/ only applies to the path, not the hostname (or IP) part of the url. You'd have to use a walk script or some sort.
Post Reply