We have been using various versions of NCSA HTTPD for a couple of years,
and the only problem I can recall like yours was on a Unixware box with
a flaky SCSI controller.
We have NCSA installed on a great variety of boxes and they've all
proven to be very stable. Compilation of some of the exotic options
for it can be a little troublesome on occasion.
There is no limit except your CPU speed and memory. As is the case
for any software. If you're going to take lots of hits (search or not),
get a fast CPU, plenty of memory, and fast disks.
Have you any experience with Server failures (traps, etc.) caused by a
large number of users trying to access Webinator? We are OK so far, but
would like to know if there is some kind of ceiling to look out for--our
concern started with our first Web server which was an OS2 version of
HTTPD which did great until we crossed about 3,000 accesses/day. I hope
the Unix based version of HTTPD is more stable.
