Table size problem?

b.sims · Post by **b.sims** » Wed Jan 23, 2002 5:57 am

While running a big run in Webinator, the following error messages are being produced (just a sample, there are more.) I guess this is a 2GB file size problem, since the html.tbl is about that size at the moment (plus I notice that bytes received is sometimes now a negative number, sometimes not). The walk continues, and it seems that inbetween errors some pages are still being received.

Is there any way round this? I am running Texis NT 64-bit on Win2000 Server

Thanks...

100 D:\Web-Root\webinator\dowalk(storepage) 1466: Date 1970-01-01 00:01:58 is out of range while processing url http://www.jamstec.go.jp/frsgc/kansoku/ ... /99kb.html

005 D:\Web-Root\webinator\dowalk(storepage) 1501: Can't read start pointers from KDBF file D:\MORPH3\texis\bigruntest\db2\html.tbl: error 13: Permission denied in the function kdbf_read_start while processing url http://www.fao.org/WAICENT/FAOINFO/AGRI ... -surv.html

100 D:\Web-Root\webinator\dowalk(storepage) 1501: Record write failed while processing url http://www.fao.org/WAICENT/FAOINFO/AGRI ... -surv.html

005 D:\Web-Root\webinator\dowalk(storepage) 1501: Can't read start pointers from KDBF file D:\MORPH3\texis\bigruntest\db2\html.tbl: error 13: Permission denied in the function kdbf_read_start while processing url http://www.fao.org/WAICENT/FAOINFO/AGRI ... ctive.html

100 D:\Web-Root\webinator\dowalk(storepage) 1501: Record write failed while processing url http://www.fao.org/WAICENT/FAOINFO/AGRI ... ctive.html

005 D:\Web-Root\webinator\dowalk(storepage) 1501: Can't read start pointers from KDBF file D:\MORPH3\texis\bigruntest\db2\html.tbl: error 13: Permission denied in the function kdbf_read_start while processing url http://www.fao.org/WAICENT/FAOINFO/AGRI ... -tech.html

100 D:\Web-Root\webinator\dowalk(storepage) 1501: Record write failed while processing url http://www.fao.org/WAICENT/FAOINFO/AGRI ... -tech.html

Post by **John** » Wed Jan 23, 2002 10:07 am

The error message indicates a permission problem, rather than a size problem. The variable which keeps track of bytes received may roll over, but that should not affect storage. What type of filesystem are you using, and what are the permissions on the html.tbl file?

You can verify that you have the 64-bit version of Texis by running texis -version, which should end with (i686-intel-winnt-64)

b.sims · Post by **b.sims** » Fri Jan 25, 2002 4:11 pm

The OS is Windows 2000 Server. The html.tbl file in the relevant database has full control permission granted to the local IUSR2. Note that this error only occurs sometimes, albeit in large blocks; the majority of pages go through normally.

I did in fact have the 32-bit Texis running due to a mix-up (although run continued after 2gig was reached). I installed 64-bit and started another run (on a different set of sites) but the same errors occur (see bottom of this post). The run is continuing however.

100 D:\Web-Root\webinator\dowalk(storepage) 1501: Record write failed while processing url http://www.ippf.org/cairo/prague1.htm

005 D:\Web-Root\webinator\dowalk(storepage) 1501: Can't read start pointers from KDBF file D:\MORPH3\texis\ercbrc\db2\html.tbl: error 13: Permission denied in the function kdbf_read_start while processing url http://www.ippf.org/regions/sar/pressreleases/2001a.htm

100 D:\Web-Root\webinator\dowalk(storepage) 1501: Record write failed while processing url http://www.ippf.org/regions/sar/pressreleases/2001a.htm

Post by **John** » Sat Jan 26, 2002 11:24 am

It's difficult to know exactly what is causing that, but it is the OS that is denying permission to write to the file. Is anything else happening during the time you get the messages?

b.sims · Post by **b.sims** » Mon Jan 28, 2002 9:10 am

It is running on an 'active' webserver, so possibly something is interfering. I will try and take a look in the logs. Are there any implications with this error, ie will the finished run still be usable? (I suppose with the exceptions of those pages that gave out errors)

Also, could you point me towards the meaning of this error message which I am also getting:

100 D:\Web-Root\webinator\dowalk(storepage) 1466: Date 1970-01-01 00:02:09 is out of range while processing url http://www.jamstec.go.jp/frsgc/public/eng/f_str1_e.html

It appears for various pages. Is the bad date coming from the external server?

Post by **mark** » Mon Jan 28, 2002 10:40 am

Yes, it should be usable with the exception of the missed pages.

The date message is caused by poorly formed "Last-Modified" information from the server that can not be properly parsed or indicates an invalid date.