To the Thunderstone staff: Happy holidays. Now, on to the question.
I am indexing customer sites with Webinator and full texis. For the most
part, these sites are located in accounts on ISPs all over the place. I
would like to walk their sites only without following links to their friends
on the same ISP.
Eg. http://www.momandpop-isp.net/~joes/
Visit mah buddy Herbs's page at http://www.momandpop-isp/~herb/
I want to walk a series of urls that I specify in "Urls.txt" while staying
within the directory specified in the URL:
Something like:
gw -dsites -j"&Url.txt" "&Url.txt"
Can I do this with Webinator? I suppose with texis and webscript I could
<sql "select Urls from ATable"></sql>
<loop $Urls>
<exec gw -j$Urls $Urls>
</exec>
</loop>
--- but it seems like gw would start, index, stop, start, index, stop...
Any suggestions?
Steve Ferda