Thunderstone Support Forums

I started an index build about 3 hours ago on a table. I just checked the directory and all I have to show for it is this:

rw------- 1 database system 74 Feb 25 12:08 xmmauth_D.btr
-rw------- 1 database system 955 Feb 25 12:08 xmmauth_P.tbl
-rw------- 1 database system 396 Feb 25 12:08 xmmauth_T.btr

the command issued was

tsql "set indexspace='/db/indexes/prodnew/';set delexp=0;set addexp='\alnum{1,99}';set addexp='[\alnum\x27\x2E]{2,99}';
set keepnoise='1';
create metamorph inverted index xmmauth on books (AUTHOR, C_PRICE, L_PRICE, QTY_AVAIL)"

What do you think the problem is.. the machine is an alpha with 4 gigs of ram and 2 CPU's. the table has 14 million rows in it.

Is the tsql process still running? It will build significant portions of the index in memory before writing them to disk, and if it needs to create temporary files it may use TMPDIR instead of the indexspace directory.

It is still running.. in fact I have one that's been running for 15 hours now and still no indexes.. no issues with the server that I can see either. Is there a way to force it to use the indexspace directory, those disks are a lot faster. I built these same indexes on a lesser machine with no problem but that was Linux Texis. Any other suggestions you may have would be appreciated. Thanks.

You can force the index process to use just a specific set of directories for temp files with addindextmp (similar to addexp):

set addindextmp='/my/tmp/dir';
set addindextmp='/my/other/tmp/dir';

Texis will only use those dirs specified by addindextmp for temp files, unless none are given, in which case indexspace (where the index is finally built) and $TMP and $TMPDIR are used, if set. The largest-free-space filesystem is used first, with a skosh factor if it's indexspace.

Keep an eye on the memory usage of the process; by default about 40% of physical RAM is used before a temp index is created (see indexmem to alter this). As long as the process is slowly growing in memory size, it's still indexing data and hasn't yet flushed to disk.

On second thought, the fact that no xmmauth.dat,.btr, and .tok files are present is suspicious. This looks like an index process died and wasn't able to completely clean up (delete index files). Are you saving stdout/stderr of the index process to a file to check for errors? If the process isn't changing size and isn't creating Tnnn* files, I'd kill it and try again, checking for errors.

I killed it and will try running it again.. It starts to build the tok file and then it dissappears. Also even though the tsql is still running, there are no signs of the process when I check the database with ltest.

Did you get any odd messages from the index build? You might also check vortex.log and maybe monitor.log.

No.. there were no odd messages nor anything in the two log files.. I just checked.. I am making another attempt at the command below to see what happens.


tsql "set indexspace='/db/indexes/prodnew/';set delexp=0;set addexp='\alnum{1,99}';set addexp='[\alnum\x27\x2E]{2,99}';
set keepnoise='1';
create metamorph inverted index xmmauth on books (AUTHOR, C_PRICE, L_PRICE, QTY_AVAIL)"

The outcome of the above command was
000 tsql ABEND: signal 11

and these files

-rw------- 1 database system 74 Feb 26 09:35 xmmauth_D.btr
-rw------- 1 database system 955 Feb 26 09:35 xmmauth_P.tbl
-rw------- 1 database system 396 Feb 26 09:35 xmmauth_T.btr

Help!!

000 tsql ABEND: signal 11

and these files

-rw------- 1 database system 74 Feb 26 09:35 xmmauth_D.btr
-rw------- 1 database system 955 Feb 26 09:35 xmmauth_P.tbl
-rw------- 1 database system 396 Feb 26 09:35 xmmauth_T.btr

Help!!