Thunderstone Support Forums

Hi,

I have two profiles, one of them is a fairly big walk and is hard to complete. I want to combine these two walk data, or databases, into one large database. I wrote a vortex script which selects the fields from db1 and inserts them to db2, is there anything else I need to do? My script goes like this:


<a name=main>
<DB=$db1>
<SQL ROW "select * from html">
<DB=$db2>
<SQL NOVARS "insert into html values($id, $New, $Visited, $Dlsecs, $Depth, $Url, $Title, $Body, $Meta)"></SQL>
</SQL>
</a>

I read in a post that you have to reindex them as well. How do you run the -index function? can you just insert "gw -index" in the function above after the SQL statement and run it with texis.exe on the browser?

I can't do a complete rewalk and put them into separate categories because it's too large to run again.

Any help is greatly appreciated, thanks in advance.

Yes, the search index on db2 needs to be updated. How depends on what version of webinator you're using. You mention gw implying version 2, but that's very old. Version 5 that uses "dowalk" to crawl the data is the current. Which are you using?

I have commercial Webinator, version 5.03. So I can use dowalk as a command in my script? What I'm doing now is creating a new script, called "merge",

<a name=main>
<DB=$db1>
<SQL ROW "select * from html">
<DB=$db2>
<SQL NOVARS "insert into html values($id, $New, $Visited, $Dlsecs, $Depth, $Url, $Title, $Body, $Meta)"></SQL>
</SQL>
dowalk
</a>

Then I open my browser to http://localhost/texispath/texis.exe/merge

and run it.

Would that acutally work?

Thanks so much for your help!

I just looked at the dowalk script and there are 3 functions related to indexing:

<a name=createindex private drop=0 meter=0>
<a name=mkindex private drop=0 force=0 type="" index schema>
<a name=updatemmindex export drop=0 meter=0>

can I just make the following changes in my script:

<a name=main>
<DB=$db1>
<SQL ROW "select * from html">
<DB=$db2>
<SQL NOVARS "insert into html values($id, $New, $Visited, $Dlsecs, $Depth, $Url, $Title, $Body, $Meta)"></SQL>
</SQL>
dowalk mkindex
</a>

and run the "merge" script directly?

See the manual about "Using dowalk" for instructions on how to invoke it. You'll want the "updateindex" entry point. See http://www.thunderstone.com/texis/site/ ... ing+dowalk

Use <exec> in your script to invoke it. See the vortex manual http://www.thunderstone.com/site/vortexman/node43.html

For maintenance jobs like you'll probably want to invoke it from the command line rather than from web. eg
texis merge/main.txt
See the vortex manual for the command line. http://www.thunderstone.com/site/vortexman/node205.html

Thanks, but I'm still a bit confused - the document you mentioned did not specify how to invoke updateindex in dowalk. Do I need to save my script as "main.txt"? I just saved it as "main" with no extension. Also, can I use the command reindex directly from the command window? Such as

texis profile=PROFILENAME dowalk/reindex.txt

The manual says how to do it. It gives an example. Then lists all of the other entries you can use. It would be something like
texis profile=PROFILENAME dowalk/updateindex.txt

My above example
texis merge/main.txt
assumes the script file is named "merge" as you've said earlier.

You can run texis from the command window (as I suggested for running your merge script). Or you can run it from another script using the <exec> call.