Page 1 of 1
request
Posted: Wed Aug 11, 1999 3:19 pm
by Thunderstone
I would like to index 37 sites. I have the 37 sites in a text file.
For
each site I would like to index every page on that site (no outside
links,
just pages on
www.thatdomain.com, is that the -j option?) and save that
site's information in it's own database, thus creating 37 databases.
Is
there a way that I can make a text batch file that lists the 37 site
walking
commands in order and then just let webinator run off the batch file and
make all the databases on the fly? Thanks.
request
Posted: Wed Aug 11, 1999 5:37 pm
by Thunderstone
-j is not necessary for you. webinator does not walk offsite unless
you tell it to do so.
You just need to create a list of 37 commands:
gw -ddbsite1
http://site1/
gw -ddbsite2
http://site2/
...
A vortex script along these lines could automate it:
<script language=vortex>
<timeout=-1></timeout>
<a name=main>
<read "sites.lst"><split "\r?\n" $ret><$sites=$ret>
<loop $sites>
<$s="http://" "[^\alnum]">
<$r="" "">
<sandr $s $r $sites>
<strfmt "-d%s" $ret><$db=$ret>
gw $db $sites
<!-- uncomment this to run the command
<exec nobr gw $db $sites></exec>$ret
-->
</loop>
</a>
</script>