unable to open table - no such table

joerg.hans.pasch · Post by **joerg.hans.pasch** » Thu May 11, 2006 4:34 am

Hi all,

after I had to stop a walk manually (because it walked more than a day without much progress) a newly startet walk gave the following error.

Thanks for your help.
Best regards
Jörg

002 /webinator/dowalk(longreport) 7081: Unable to open table C:\Program Files\Thunderstone Software\Webinator\texis\gf\db2\SYSCOLUM in the function opendbtbl 115 /webinator/dowalk(longreport) 7081: No such table: SYSCOLUMNS in the database: C:\Program Files\Thunderstone Software\Webinator\texis\gf\db2\ 000 /webinator/dowalk(longreport) 7081: SQLPrepare() failed with -1 in the function prepntexis

Post by **mark** » Thu May 11, 2006 11:19 am

By "manual" do you mean by hitting the "stop" button or forcefully killing it with task manager or such?

What does the walk status say?

Your c: drive isn't full is it?

What files ARE in C:\Program Files\Thunderstone Software\Webinator\texis\gf\db2 ?

joerg.hans.pasch · Post by **joerg.hans.pasch** » Fri May 12, 2006 4:56 am

Hi Mark.

1- with manually I mean the stop button on the walk page.
2- more than 80Gb space left.
3- Looks like any other working profile. (db folder, tables and summary files. All are here and not corrupt)

Now, what's strange is that this night the walk worked fine.
Last complete walk: 2006-05-12 02:20:10 (took 20 minutes 3 seconds)
Success. 3,645 pages (418,743,512 bytes) (251 errors) (352 duplicates)
But that only since we set back the system to "URL page". If we set this to "URL URL", the error occur again.

Regards
Jörg

Post by **mark** » Fri May 12, 2006 5:14 pm

Do you mean "page url" (there's no "url page")?
Are you doing a new or refresh walk? Try new. Start it manually instead of waiting for a scheduled start.

joerg.hans.pasch · Post by **joerg.hans.pasch** » Thu May 18, 2006 4:51 am

Hi.
Sorry fot the delay.

Yep, sorry, it's page url.
And yes, I tryed a new walk, and start it manualy.
After a few tests, I seams that url url don't work, and if we set it back to page url, everything is fine.

Are on url url some special http header needed? The url we set sends out the lines one by one (using flush system to allow to begin working eralier) and not as one data block.

You see what I mean?

best regards
Jörg

Post by **John** » Thu May 18, 2006 8:53 am

The crawler will download the entire URL URL and process it before it starts crawling. If it is hitting the fetch timeout that might cause some issues. You may want to modify the scripts some to have a Base URL or two that it can start crawling on, and then a separate process to populate the todo table from the URL URL.

joerg.hans.pasch · Post by **joerg.hans.pasch** » Thu May 18, 2006 9:34 am

Hi John,

thanks for your help!
Modifying the walkall script might be an alternative ...
But before we try that, maybe I can give some more background on what we try to do. If you say, that is not possible, then we do not need to change the script:

Since we use a CMS with dynamic page urls and we found that letting Webinator crawl the whole site by itself leads to unwanted results, we first checked the "Page URL" option. Using that, we fed a long - as we call it - "spider page" into Webinator, with all the page urls we needed to be indexed (> 3000). The first time, that worked fine. But all rewalks (no matter if refresh or renew) do not make Webinator index the new pages that had been added in the meantime.

Therefore we switched to "URL URL" because that sounded more promising. Documentation says "the URL will be refetched each time a rewalk is started".
If it did that without errors, we would be happy.
Do you think this is possible?

Kind regards
Jörg

Post by **John** » Thu May 18, 2006 10:07 am

If you add the spider page as Base URL and a Watch URL it will be rechecked on every walk.

joerg.hans.pasch · Post by **joerg.hans.pasch** » Mon May 22, 2006 8:15 am

Hi John.

Hmm, I think this is not a solution as we can not spider the site from the base url. We need to pass to webinator an text file with the urls inside.
If we pass this file as "page url", this works, but do not index new pages listed in the file.
If we use "url url" for this file, the system returns an error (the one we post at the beginning)
I tested both with rewalk or new.

Any other hint about this?

Many thanks for any answer.
Best regards Jörg.

Post by **mark** » Mon May 22, 2006 12:41 pm

I'm not able to replicate anything like that. Make sure you have the latest dowalk and search scripts from the website.