Help with page rendering of Vortex script output in browser window

[barry.marcus]

This is most likely NOT a Vortex issue per se, but I thought I'd seek some help here anyway, just in case someone can offer some guidance. So far I've not had much luck finding a fix elsewhere...

A section of one of our scripts performs a number of intensive calculations and data manipulations that, depending on the query, can take a fair amount of time. Sometimes the total time for all the operations behind a page request can take upwards of 10 to 15 minutes to complete before the page is completely rendered in the browser. Believe it or not, we're OK with that because, using the Apache web server on a Linux OS, the output of each operation is rendered as it's available. That is, the code is structured (or so we thought) so that the user sees the page slowly, but steadily, building.

We've ported the app to run on Windows Server using the IIS 7.0 web server. Everything runs fine except that, for those calculation/data intensive pages described above, something somewhere is buffering the output of the script until the entire page is available before it is rendered in the browser. While we're OK with the page slowly building, we're NOT OK with users staring at a blank page for 15 minutes before seeing any of their results.

The code is NOT written in a way that would require the browser to have a full page before being able to render. That is, the entire output for the page is NOT enclosed in one large table, for example. Granted, there are tables involved in the formatting of the output... The output of each separate calculation (each of which takes less than a minute) is in it's own table. And yet in this environment, nothing renders until all of the output is available. I've made liberal use of <flush> to no avail. And as I said, the behavior is fine on Apache/Linux. So I suspect that this is an IIS 7.0/Windows Server issue. Can anyone offer suggestions as to what I might tweak in IIS, or offer any ideas on a general approach to structuring the output of a page that takes a fair amount of time to complete?

Thanks in advance.

[Kai]

Since you're <flush>ing everywhere and still seeing a delay in page loading, it's probably something downstream of Vortex. Are you using CGI, or the Texis ISAPI filter, under Windows?

[jason112]

I don't have experience with IIS7 & buffering, but it looks like part b) mentioned in the 2nd message might work: http://forums.iis.net/t/1151673.aspx

We do have experience with managing long-running tasks started from a webpage - namely, our walks.

Rather than performing the calculations in the web request, you could <exec> a script that does the calculations and writes the output to disk. The webpage can refresh itself periodically and display the contents of the file.

This adds a little more overhead as you'll occasionally have to clean up the pages on disk, but you'll then have a setup that's completely independent of any caching/buffering web servers.

[barry.marcus]

Kai - We're using CGI.
Jason - Thanks for the link... I'll take a look at it.

[Kai]

Ok, if you're using CGI then there's no issue of the additional ISAPI layer potentially buffering (either in the Texis module or IIS). I've seen web servers arbitrarily buffer output from CGI programs like Vortex before (even with copious <flush>es); the only solution I found was to find and tune the appropriate web server buffer settings to get the web server to not buffer at all, which is not always possible. Or better yet, as Jason suggested, run long-term scripts in the background to disk and just display that output in a foreground (CGI) script.