Multiple Crawling At Once

Post by **Thunderstone** » Fri Dec 20, 1996 3:15 pm

Is there a way to start ONE process of GW running and then keep adding to
it's todo list on the fly? We are running GW here right now to do many
local domains and would like to run one process instead of multiple ones
and just "feed it" a list of sites to visit....

Thanks,

Paul

--
Out the modem, through the SPARC, down the T3, off the router, past the
frame-relay... nothing but Net.

Post by **Thunderstone** » Fri Dec 20, 1996 4:15 pm

You can't feed it on the fly. Just run multiple gw's in sequence:
gw http://www.firstsite.com
gw http://www.secondsite.com
...
gw -index