Missing HTTP Response

mjacobson · Post by **mjacobson** » Mon May 23, 2005 8:35 am

What would cause the error message, "Missing HTTP Response" during a Webinator index? I can browse to the pages that are returning this response. I get a 200 when I do the following from a command line on the Webinator's server

telnet servers_name 80
HEAD /doc_file_path HTTP/1.0

I have turned up the Verbosity to 6 and get no other indications and I don't see anything in the log files.

Post by **Kai** » Mon May 23, 2005 11:30 am

Some web servers seem not to handle Keep-Alive properly, and can give an empty response on the first or later request of a Keep-Alive connection. Set Max Requests to 1 under All Walk Settings, or try walking with Net Mode set to Sys.

mjacobson · Post by **mjacobson** » Mon May 23, 2005 11:46 am

I can not find an option for "Max Request". Is this a new option. I am running 5.1.12-Unix dowalk and search scripts dated 04/06/2005.

I tried the Net Mode but got the same results.

Post by **mark** » Mon May 23, 2005 12:01 pm

Max requests appeared in version 5.1.13. You can get the latest scripts from the website under the webinator scripts page.

legedza.henry · Post by **legedza.henry** » Thu Mar 29, 2007 2:36 am

I have just started receiving these same errors. I am running 5.1.46

I have tried reducing the Max Request from 100 to 1 and there was no improvement.

This has not happened before - is there some setting that may have been set on the web server that could have caused this to begin happening?

Post by **Kai** » Thu Mar 29, 2007 10:03 am

What version of the Texis executable are you using? Run texis -version from the Webinator install dir and post that output.

legedza.henry · Post by **legedza.henry** » Thu Mar 29, 2007 6:46 pm

Version is: 5.01.1149606138 20060607

Post by **Kai** » Mon Apr 02, 2007 12:03 pm

Ok, that should have included the known Keep-Alive fixes.

Open a tech support ticket and send us the version, URL that this happens on, Base URL and your walk settings. We'll need to try reproducing it to find a solution.

legedza.henry · Post by **legedza.henry** » Mon Apr 02, 2007 7:20 pm

I ran a reindex with NetMode set to System and this worked with no Keep-Alive errors, although it did slow down the reindex process considerably.

I guess my concern is what would have caused these errors to occur all of a sudden when there have been no changes to the servers being indexed or the server running Webinator.

Post by **Kai** » Thu Apr 05, 2007 12:25 pm

If these `Missing HTTP response' errors happened only recently, then it's nearly certain that something changed recently to cause them, ie. a Webinator upgrade, profile changes (hitting new content), server content or settings change, network or firewall changes, etc.

With Max Requests set to 1, Webinator is not even attempting Keep-Alive (it explicitly sets Connection: close); the `Missing HTTP response' message mentions `broken Keep-Alive' only because that is a common cause of the error.