Thunderstone Support Forums

I would like to exclude all IP address from being crawled. I am using gw and in the options file I added the following regexp x/>>=\digit{3}\. Is there a flag or some other way to keep webinator from walking any links that are IP address based?

There's no option or such designed for doing that. -x/ only applies to the path, not the hostname (or IP) part of the url. You'd have to use a walk script or some sort.