Huge vortex.log

[john208]

My vortex.log grew to several GB and filled my c:\ drive. I had to delete it to free up the space. Is there a way to limit the size or move it to a different drive?

The following errors were in the db1.long file. Can you tell me what's wrong?

<pre>
Webinator Walk Report for www.mysite.org

Creating database W:\WebinatorDB/www.mysite.org/db1...Done.
Walk started at 2006-09-02 17:41:58 (by user)
Verbosity set to 4
JavaScript walking not enabled by current license
HTTPS walking enabled
Start fetching at http://www.mysite.org
http://www.mysite.org
Ignore urls containing any of the following:
/errors/
/Scripts/

started 1 new (15392) on http://www.mysite.org
Process memory limit exceeded (current: 26,042,368, limit: 25,000,000)
1287 pages fetched (52,671,825 bytes) from http://www.mysite.org
started 1 (1036) Resume 44f9faa61e

006 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2392: Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db1\counts.tbl: No file write permission in the function kdbf_put while processing url http://www.mysite.org/calendars/block.a ... 80:164:156

000 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2392: Could not write updated record in the function update while processing url http://www.mysite.org/calendars/block.a ... 80:164:156

006 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2388: Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db1\counts.tbl: No file write permission in the function kdbf_put while processing url http://www.mysite.org/calendars/block.a ... &Template=

000 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2388: Could not write updated record in the function update while processing url http://www.mysite.org/calendars/block.a ... &Template=

006 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2392: Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db1\counts.tbl: No file write permission in the function kdbf_put while processing url http://www.mysite.org/calendars/block.a ... &Template=

000 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2392: Could not write updated record in the function update while processing url http://www.mysite.org/calendars/block.a ... &Template=

006 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2392: Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db1\counts.tbl: No file write permission in the function kdbf_put while processing url http://www.mysite.org/calendars/block.a ... &Template=

000 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(logerror) 2392: Could not write updated record in the function update while processing url http://www.mysite.org/calendars/block.a ... &Template=
Dispatcher stopping by request. May take up to 65 seconds to stop.
Cancelled by user: 2006-09-02 23:54:57 <br>
End of report.<br>

[John]

You can specify the location of the log file in the texis.cnf file.

It looks as if there isn't sufficient permissions on the database files for the user running the walk. If you are running the cgi-bin texis.exe rather than the ISAPI filter, or don't have texis registered as a service it will run as the IUSR_ account, which typically has very limited permissions.

[john208]

This is run by a scheduled task. The task account should have enough permissions. It runs several new walks fine, and then has this problem once in a while. We are using cgi-bin but is IUSR even used by a scheduled task? How could the permissions error be intermittent?

Thanks, John

[john208]

Any ideas why the scheduled task lacks permissions? The IUSR already has full permissions to the database folders.

Thanks, John

[John]

Is that a windows scheduled task or a texis scheduled task. Is texis registered as a service?

If the texis monitor, which runs texis scheduled tasks is stopped, and a cgi-bin texis is invoked, it will try and start the texis service, and if that fails will start the process itself, which would be as the IUSR_ account.

[john208]

I see a Texis Monitor service, but it's on manual and is not started. Is that the service you are asking about?

We use a Windows scheduled task. Like I said, the IUSR already has full permissions to the database folders.

When I look at the task using ProcessExplorer (www.sysinternals.com), it shows as the account that runs the Windows scheduled task.

[mark]

Yes, the texis monitor service should be running. It should be automatic on system startup.

Does the account that runs the scheduled task have permissions?

[john208]

When I tried to start the service, it says "could not start the Texis Monitor on Local Computer. Error 230: the pipe state is invalid"

The account that runs the scheduled task has full permission.

[John]

I'm not sure what would be causing the pipe state message.

Is the W: drive where the database is local? The other potential cause would be if you have a process such as a backup that locks files from write while it backs them up, and denies write permission.

[john208]

I am still getting these errors. Backup is not running and I don't think any of my processes is locking the files. I gave everyone write permission to the database folder as a test, and I am still getting the same errors.

The new walk is just getting stuck on 16,969 pages. The number of pages doesn't change after several minutes.

0 pages in todo
16,969 pages scheduled to be refreshed
0 pages visited in the last hour (0 success/0 failed)
16,969 pages in index


Pages recently walked
16966 pages (370,066,316 bytes) so far.
358 errors so far.
24115 duplicate pages so far.

Can you think of why this is happening?

Here are more errors.

John

100 2006-09-09 11:50:11 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 16300 in the function tx_prw_endiothread
100 2006-09-09 12:27:51 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 9324 in the function tx_prw_endiothread
100 2006-09-09 12:32:38 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Document not found: http://www.mysite.org/athletic/index.asp returned code 404 (Not Found)
100 2006-09-09 12:51:57 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 14844 in the function tx_prw_endiothread
006 2006-09-09 12:52:17 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 12:52:17 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 12:52:17 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 12:52:17 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 12:54:05 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 12:54:05 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 12:54:05 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 12:54:05 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 13:02:08 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 17104 in the function tx_prw_endiothread
100 2006-09-09 13:11:09 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 13352 in the function tx_prw_endiothread
006 2006-09-09 13:16:35 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:16:35 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 13:16:35 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:16:35 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 13:17:21 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:17:21 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 13:17:21 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:17:21 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 13:18:30 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 27372 in the function tx_prw_endiothread
100 2006-09-09 13:26:13 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 26476 in the function tx_prw_endiothread
100 2006-09-09 13:36:02 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 6880 in the function tx_prw_endiothread
006 2006-09-09 13:40:53 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:40:53 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 13:40:53 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:40:53 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 13:44:22 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 15436 in the function tx_prw_endiothread
006 2006-09-09 13:44:42 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:44:42 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 13:44:42 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:44:42 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 13:51:22 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 15412 in the function tx_prw_endiothread
006 2006-09-09 13:56:22 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:56:22 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 13:56:22 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 13:56:22 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 13:58:14 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 3812 in the function tx_prw_endiothread
100 2006-09-09 14:05:20 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 19188 in the function tx_prw_endiothread
006 2006-09-09 14:06:01 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:06:01 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 14:06:01 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:06:01 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 14:09:32 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Document not found: http://www.mysite.org/gp/reader/1595140 ... 93-5894530 returned code 404 (Not Found)
100 2006-09-09 14:12:16 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 28392 in the function tx_prw_endiothread
006 2006-09-09 14:12:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:12:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 14:12:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:12:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 14:15:51 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:15:51 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 14:20:04 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 27376 in the function tx_prw_endiothread
006 2006-09-09 14:26:44 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:26:44 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 14:30:02 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 12220 in the function tx_prw_endiothread
100 2006-09-09 14:44:33 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 14612 in the function tx_prw_endiothread
006 2006-09-09 14:49:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:49:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 14:49:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 14:49:46 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 14:51:25 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 23468 in the function tx_prw_endiothread
100 2006-09-09 14:57:44 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 26816 in the function tx_prw_endiothread
100 2006-09-09 15:04:11 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 4920 in the function tx_prw_endiothread
006 2006-09-09 15:06:49 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 15:06:49 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
006 2006-09-09 15:06:49 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Cannot write to KDBF file W:\WebinatorDB\www.mysite.org\db2\counts.tbl: No file write permission in the function kdbf_put
000 2006-09-09 15:06:49 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk Could not write updated record in the function update
100 2006-09-09 15:10:25 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 6884 in the function tx_prw_endiothread
100 2006-09-09 15:17:07 C:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk:4837: Terminating stdin I/O thread for process 15276 in the function tx_prw_endiothread
000 2006-09-09 18:06:54 /Webinator/search:729: Timeout
002 2006-09-09 21:42:04 /00&rprox=500&rdfreq=500&rwfreq=500&rlead=500&sufs=0&x=0&y=0: Cannot open source file: No such file or directory