Hardware needed for huge text indexes?

james120 · Post by **james120** » Tue Aug 31, 2004 7:45 pm

I am interested in doing full-text indexing, and then performing sophisticated queries, on 100-300 gig of data. So far I have not been able to find any benchmarks for databases of this size. Any input would be appreciated.

Post by **John** » Wed Sep 01, 2004 10:26 am

We do have several customers with 100+GB of data. They generally run on 2 or 4 CPU servers, with fast disks and plenty of RAM. Depending on the queries such a server should support several hundred thousand queries a day.

james120 · Post by **james120** » Tue Sep 07, 2004 2:29 pm

I note that the specifications for Texis say that tables are limited to 2 gig on 32 bit systems. What about systems that can do either 32 or 64? Are AMD Opterons and Xeon Noconas 32-bit or 64-bit for purposes of table size?

Post by **John** » Tue Sep 07, 2004 2:41 pm

It's actually up to the OS rather than the processor. 64-bit table sizes are supported on most operating systems these days.

james120 · Post by **james120** » Tue Sep 07, 2004 3:37 pm

Thanks for the info. So, I assume that current Linux kernals (2.4 or 2.6) do support this, but could you confirm this for me so I don't make a serious OS choice mistake? I am currently planning on going with Fedora Core 2 (based on 2.6).

Post by **John** » Tue Sep 07, 2004 3:41 pm

Yes, Linux 2.4 and higher does support 64-bit file sizes.