Thunderstone Support Forums

What would cause the error message, "Missing HTTP Response" during a Webinator index? I can browse to the pages that are returning this response. I get a 200 when I do the following from a command line on the Webinator's server

telnet servers_name 80
HEAD /doc_file_path HTTP/1.0

I have turned up the Verbosity to 6 and get no other indications and I don't see anything in the log files.

Some web servers seem not to handle Keep-Alive properly, and can give an empty response on the first or later request of a Keep-Alive connection. Set Max Requests to 1 under All Walk Settings, or try walking with Net Mode set to Sys.

I can not find an option for "Max Request". Is this a new option. I am running 5.1.12-Unix dowalk and search scripts dated 04/06/2005.

I tried the Net Mode but got the same results.

Max requests appeared in version 5.1.13. You can get the latest scripts from the website under the webinator scripts page.

I have just started receiving these same errors. I am running 5.1.46

I have tried reducing the Max Request from 100 to 1 and there was no improvement.

This has not happened before - is there some setting that may have been set on the web server that could have caused this to begin happening?

What version of the Texis executable are you using? Run texis -version from the Webinator install dir and post that output.

Version is: 5.01.1149606138 20060607

Ok, that should have included the known Keep-Alive fixes.

Open a tech support ticket and send us the version, URL that this happens on, Base URL and your walk settings. We'll need to try reproducing it to find a solution.

I ran a reindex with NetMode set to System and this worked with no Keep-Alive errors, although it did slow down the reindex process considerably.

I guess my concern is what would have caused these errors to occur all of a sudden when there have been no changes to the servers being indexed or the server running Webinator.

If these `Missing HTTP response' errors happened only recently, then it's nearly certain that something changed recently to cause them, ie. a Webinator upgrade, profile changes (hitting new content), server content or settings change, network or firewall changes, etc.

With Max Requests set to 1, Webinator is not even attempting Keep-Alive (it explicitly sets Connection: close); the `Missing HTTP response' message mentions `broken Keep-Alive' only because that is a common cause of the error.