reindexing error

jkj2001 · Post by **jkj2001** » Thu Apr 17, 2003 12:08 pm

Hi guys,

We got this error on one of our indexes:

011 Can't alloc 97224009 bytes of memory via fdbi_allocbuf: Not enough space in
the function wtix_addmerge
000 Abandoning index /reindex/bankofamerica/tbldoc/idxmtbldoc_DOCTEXT due to err
ors
000 SQL failed

Another of our indexes failed with this:

011 Can't alloc 64577264 bytes of memory: Not enough space in the function putkloc
000 Abandoning index /reindex/bankofamerica/tbldoc/idxmtbldoc due to errors

We were running these commands from the bourne shell:

create metamorph inverted index idxmtbldoc_DOCTEXT on mytable(somefield)

create metamorph inverted index idxmtbldoc on mytable(somefield\someotherfield\anotherfield)

Admittedly, the DOCTEXT field is pretty big; its .dat file is maybe 3 GB in size after a successful reindex, but we *have* reindexed it successfully before, and the field has essentially been the same size over the last few reindexes we've run. Doesn't make sense why it would blow up now.

Maybe there are some Solaris (that's what we're using, with 4 GB of RAM) parameters we need to tune? The drive this database is sitting on is only 25% full, so I don't think it's a case of our running out of disk space.

We're using the Solaris version of Thunderstone, version 4.00.1019027849. Thanks for your help.

Post by **Kai** » Thu Apr 17, 2003 1:31 pm

Disk space seems adequate; up to 2x the current index usage may be needed to update an index, plus room for new data.

Do you have any ulimits set? Run ulimit -a from the same prompt you run the index create from.

jkj2001 · Post by **jkj2001** » Thu Apr 17, 2003 2:17 pm

Thanks for the help, Kai. Here's what we get with ulimit:

core file size (blocks) unlimited
data seg size (kbytes) unlimited
file size (blocks) unlimited
open files 256
pipe size (512 bytes) 10
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 29995
virtual memory (kbytes) unlimited

Post by **mark** » Thu Apr 17, 2003 3:44 pm

You have 4GB ram, but how much is available? Do you have more than one create index running at the same time (as opposed to sequentially)? How big does the texis process get before it runs out of mem? Maybe you need to set indexmem to something other than the default of 40% of system ram. See http://thunderstone.master.com/texis/ma ... q=indexmem

jkj2001 · Post by **jkj2001** » Thu Apr 17, 2003 4:17 pm

We just tried to create one of the indexes again, with the same result. Things went along for about two hours before the trouble came.

Last night we ran the reindexes sequentially, with nothing to slow them down. Today we had a second database reindexing at the same time, but it went through with no problems.

I can't understand why this is happening all of a sudden. As I mentioned at the beginning of the thread, we've created these "problem" indexes several times before, with almost exactly the same data, and had no problems.

Is it possible for texis to create large temp files outside of the database's directory? I'm referring to the T* files which appear when large reindexing jobs run their course. Can't think of why that would happen suddenly here, but it's something to investigate I suppose.

Did anything in the ulimit -a jump out at you? Thanks again.