Dealing with table size limitations

Post Reply
b.sims
Posts: 99
Joined: Fri Oct 26, 2001 10:40 am

Dealing with table size limitations

Post by b.sims »

While going through the Texis specifications, I note that maximum table size for a 32-bit machine is 2GB. I wondered about people's experience in getting around this in common use. In my case, the use will be for indexing a large number of web pages. My test run of 10,000 pages added up to 100MB; I guess this means I will be able to have 200,000 with a 2GB limit.

I thought of several ways to do this, for example by dividing the pages up into categories and then running the search on them together.

Has anyone any experience of doing this sort of thing? I suppose I could also limit the amount of data received for each page, although this would affect functionality.

Help much appreciated,
Benjamin Sims
User avatar
mark
Site Admin
Posts: 5519
Joined: Tue Apr 25, 2000 6:56 pm

Dealing with table size limitations

Post by mark »

Most people switch to a platform where we have large file support such as Solaris 2.6 or above, Tru64 on the alpha, Linux 2.4 or above on Intel, FreeBSD, or Windows NT/2K/XP.

Another possibility to perform the same query against each table and sort the results together in a ram table or such. This obviously requires more horepower because you would be doing N queries and a sort for every user query.
b.sims
Posts: 99
Joined: Fri Oct 26, 2001 10:40 am

Dealing with table size limitations

Post by b.sims »

So, with a Windows Texis license I am thinking about an Itanium machine?
User avatar
John
Site Admin
Posts: 2622
Joined: Mon Apr 24, 2000 3:18 pm
Location: Cleveland, OH
Contact:

Dealing with table size limitations

Post by John »

No, the standard version of Texis for Windows uses 32-bit files. We do have a version that uses 64-bit files. The processor can be a Pentium, or other Intel compatible processor that supports Windows.
John Turnbull
Thunderstone Software
b.sims
Posts: 99
Joined: Fri Oct 26, 2001 10:40 am

Dealing with table size limitations

Post by b.sims »

So I could use larger than two-gig files with this? What is the cost involved?

If I use Linux rather than Windows for the server, will I see a substantial gain in performance?
User avatar
John
Site Admin
Posts: 2622
Joined: Mon Apr 24, 2000 3:18 pm
Location: Cleveland, OH
Contact:

Dealing with table size limitations

Post by John »

Yes, you could use larger than 2-Gig files with that version for Windows, or the Linux 2.4 version. On the same hardware you should see similar performance regardless of the Linux or Windows, especially for long running tasks such as loading data and creating indexes. If you are doing invoking a lot of short tasks, such as queries then Linux may show better performance, but it is unlikely to be substantial.
John Turnbull
Thunderstone Software
b.sims
Posts: 99
Joined: Fri Oct 26, 2001 10:40 am

Dealing with table size limitations

Post by b.sims »

Do I need to buy a different license for the larger than two-gig files? At the moment I have ordered standard Texis, automatic growth.
User avatar
John
Site Admin
Posts: 2622
Joined: Mon Apr 24, 2000 3:18 pm
Location: Cleveland, OH
Contact:

Dealing with table size limitations

Post by John »

It is considered a different platform, and the files are not compatible. If your order has not been processed yet then you should contact your salesman, and you can change the platform. The price will be the same.

If you already have one platform you can order an additional platform, which is available at a low fixed cost and provides the same license limits you already have.
John Turnbull
Thunderstone Software
Post Reply