Thunderstones webservice data load times out

Post Reply
david.huber0
Posts: 22
Joined: Wed Mar 03, 2010 5:47 am

Thunderstones webservice data load times out

Post by david.huber0 »

Hello,

we are trying to send to thunderstone, through the data load webservice, many documents.

After about 5000 documents send the webservice times out. The document are sent one by one after the werbservice returns. After monitoring the texis process it appears that it uses a lot of cpu and memory. Furhtermore it seems that the process isn't able to free memory accurately while using the webservice.

For information the webservice client is a .net application.

Any idea?
User avatar
jason112
Site Admin
Posts: 347
Joined: Tue Oct 26, 2004 5:35 pm

Thunderstones webservice data load times out

Post by jason112 »

Are you seeing requests slow down with increasing times, or does it suddenly hang with a request & time out?

Once it starts timing out, what gets it to not time out?

Does it start timing out on the same document, or is it somewhat randomized?

> Furhtermore it seems that the process isn't able to
> free memory accurately while using the webservice.

What makes you say that? If you're sending the documents individually, then each request should be a separate process and return very quickly, so freeing memory should be irrelevant.
david.huber0
Posts: 22
Joined: Wed Mar 03, 2010 5:47 am

Thunderstones webservice data load times out

Post by david.huber0 »

> Are you seeing requests slow down with increasing
> times, or does it suddenly hang with a request & time
> out?

it suddenly hang with a request & time out

the time out is randomized.

Is there a max size for the web service request?
Because my application may send big files.
User avatar
jason112
Site Admin
Posts: 347
Joined: Tue Oct 26, 2004 5:35 pm

Thunderstones webservice data load times out

Post by jason112 »

There is no limit to the size that can be uploaded.

Define 'big': 1mb, 1gb, 1tb? Can you tell if it tends to hang on a 'big' file, or does it sometimes hang on smaller ones?
david.huber0
Posts: 22
Joined: Wed Mar 03, 2010 5:47 am

Thunderstones webservice data load times out

Post by david.huber0 »

the bigger files are about 20-30 MB

But sometimes the webservice hang on 1Kb files
User avatar
jason112
Site Admin
Posts: 347
Joined: Tue Oct 26, 2004 5:35 pm

Thunderstones webservice data load times out

Post by jason112 »

Anything in monitor.log or vortex.log from around the time that the timeouts occur?
kamal.oulhadj
Posts: 15
Joined: Mon Mar 01, 2010 7:53 am

Thunderstones webservice data load times out

Post by kamal.oulhadj »

I work on the same project of David Huber
in the file monitor.log we have the following logs in indexing documents of size 20-30 MB.


200 2010-03-12 10:37:14 (3512) 2010-03-12 10:37:14 (pid 3512) in the function Timestamp
002 2010-03-12 10:38:45 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
200 2010-03-12 10:39:01 (1480) Contacting license server
006 2010-03-12 10:39:14 (1480) Timeout sending data to 127.0.0.1:16141 in the function htbuf_flush
006 2010-03-12 10:39:23 (1480) Cannot connect to ls.thunderstone.com:80: Socket timeout
000 2010-03-12 10:39:23 (1480) Could not contact license server
002 2010-03-12 10:40:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
200 2010-03-12 10:41:14 (2140) 2010-03-12 10:41:14 (pid 2140) in the function Timestamp
006 2010-03-12 10:42:11 (1480) Timeout sending data to 127.0.0.1:14864 in the function htbuf_flush
018 2010-03-12 10:42:11 (1480) Cannot terminate process 3096: Access is denied in the function TXpkill
002 2010-03-12 10:42:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
002 2010-03-12 10:44:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
002 2010-03-12 10:46:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
002 2010-03-12 10:48:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
002 2010-03-12 10:50:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
002 2010-03-12 10:52:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
200 2010-03-12 10:54:30 (1480) Contacting license server
002 2010-03-12 10:54:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
006 2010-03-12 10:54:51 (1480) Cannot connect to ls.thunderstone.com:80: Socket timeout
000 2010-03-12 10:54:51 (1480) Could not contact license server
002 2010-03-12 10:56:46 (3364) Unable to open table D:\texis\data1\APCES\db1\SYSMETAI in the function opendbtbl
Post Reply