Combine multiple databases/profiles

[edev]

Hi,

I have two profiles, one of them is a fairly big walk and is hard to complete. I want to combine these two walk data, or databases, into one large database. I wrote a vortex script which selects the fields from db1 and inserts them to db2, is there anything else I need to do? My script goes like this:


<a name=main>
<DB=$db1>
<SQL ROW "select * from html">
<DB=$db2>
<SQL NOVARS "insert into html values($id, $New, $Visited, $Dlsecs, $Depth, $Url, $Title, $Body, $Meta)"></SQL>
</SQL>
</a>

I read in a post that you have to reindex them as well. How do you run the -index function? can you just insert "gw -index" in the function above after the SQL statement and run it with texis.exe on the browser?

I can't do a complete rewalk and put them into separate categories because it's too large to run again.

Any help is greatly appreciated, thanks in advance.

[mark]

Yes, the search index on db2 needs to be updated. How depends on what version of webinator you're using. You mention gw implying version 2, but that's very old. Version 5 that uses "dowalk" to crawl the data is the current. Which are you using?

[edev]

I have commercial Webinator, version 5.03. So I can use dowalk as a command in my script? What I'm doing now is creating a new script, called "merge",

<a name=main>
<DB=$db1>
<SQL ROW "select * from html">
<DB=$db2>
<SQL NOVARS "insert into html values($id, $New, $Visited, $Dlsecs, $Depth, $Url, $Title, $Body, $Meta)"></SQL>
</SQL>
dowalk
</a>

Then I open my browser to http://localhost/texispath/texis.exe/merge

and run it.

Would that acutally work?

Thanks so much for your help!

[edev]

I just looked at the dowalk script and there are 3 functions related to indexing:

<a name=createindex private drop=0 meter=0>
<a name=mkindex private drop=0 force=0 type="" index schema>
<a name=updatemmindex export drop=0 meter=0>

can I just make the following changes in my script:

<a name=main>
<DB=$db1>
<SQL ROW "select * from html">
<DB=$db2>
<SQL NOVARS "insert into html values($id, $New, $Visited, $Dlsecs, $Depth, $Url, $Title, $Body, $Meta)"></SQL>
</SQL>
dowalk mkindex
</a>

and run the "merge" script directly?

[mark]

See the manual about "Using dowalk" for instructions on how to invoke it. You'll want the "updateindex" entry point. See http://www.thunderstone.com/texis/site/ ... ing+dowalk

Use <exec> in your script to invoke it. See the vortex manual http://www.thunderstone.com/site/vortexman/node43.html

For maintenance jobs like you'll probably want to invoke it from the command line rather than from web. eg
texis merge/main.txt
See the vortex manual for the command line. http://www.thunderstone.com/site/vortexman/node205.html

[edev]

Thanks, but I'm still a bit confused - the document you mentioned did not specify how to invoke updateindex in dowalk. Do I need to save my script as "main.txt"? I just saved it as "main" with no extension. Also, can I use the command reindex directly from the command window? Such as

texis profile=PROFILENAME dowalk/reindex.txt

[mark]

The manual says how to do it. It gives an example. Then lists all of the other entries you can use. It would be something like
texis profile=PROFILENAME dowalk/updateindex.txt

My above example
texis merge/main.txt
assumes the script file is named "merge" as you've said earlier.

You can run texis from the command window (as I suggested for running your merge script). Or you can run it from another script using the <exec> call.