request

Post by **Thunderstone** » Wed Aug 11, 1999 3:19 pm

I would like to index 37 sites. I have the 37 sites in a text file.
For
each site I would like to index every page on that site (no outside
links,
just pages on www.thatdomain.com, is that the -j option?) and save that
site's information in it's own database, thus creating 37 databases.
Is
there a way that I can make a text batch file that lists the 37 site
walking
commands in order and then just let webinator run off the batch file and

make all the databases on the fly? Thanks.

Post by **Thunderstone** » Wed Aug 11, 1999 5:37 pm

-j is not necessary for you. webinator does not walk offsite unless
you tell it to do so.

You just need to create a list of 37 commands:
gw -ddbsite1 http://site1/
gw -ddbsite2 http://site2/
...

A vortex script along these lines could automate it:
<script language=vortex>
<timeout=-1></timeout>
<a name=main>
<read "sites.lst"><split "\r?\n" $ret><$sites=$ret>
<loop $sites>
<$s="http://" "[^\alnum]">
<$r="" "">
<sandr $s $r $sites>
<strfmt "-d%s" $ret><$db=$ret>
gw $db $sites

</loop>
</a>
</script>