Trying to gauge system resources used by Webinator

[Thunderstone]

Just doing a little stress-testing with webinator to gauge what
the full-blown commercial package might do...

I've indexed 9,000+ pages and by using the "top" command and the Irix
"gr_osview" command (described at
http://reality.sgi.com/cgi-bin/getman/?gr_osview ) it's appearing to
me that while a search that matches perhaps thousands of pages uses
precious little of the machine, the DISPLAYING of each page of matches
(10 matches shown at a time) takes up 70% and often more of the CPU for
up to maybe 2 seconds per page - and this is only the invocation of the
"texis" CGI and does not include the resources taken up by the web server
itself (and the browser is indeed on a separate machine). If two people
were to do this simultaneously, obviously the machine would be maxed out.

One might wonder if the aforementioned chunk of resources is high,
low, or to be expected from this type of application. I couldn't
tell ya.

So I'm looking for ideas on how to see/investigate further how the
impact of PRESENTING the results of a search with Thunderstone
software might compare against presenting similar results using
"compiled" (in the Netscape LiveWire sense) JavaScript. Since, from a
previous posting here, I understand that I can't put SQL queries into my
JavaScript so that it will search a Vortex database such as
Webinator's, I could instead build a similar Informix database and
build JavaScript to present records from THIS database. Would this
be a fair test of the resources used to present the results of a database
search? Or am I being naively simplistic or even entirely off base?

On a different note, are there any editing tools similar to Visual
JavaScript (described at
http://developer.netscape.com/library/d ... s/vjs.html )
which would provide a graphical editor for the creation of texis
scripts? Texis is like HTML in that if you know it, a true hacker
don't need no fancy-schmancy editor. Still, there's something to be
said for the likes of the HTML editors FrontPage, Dreamweaver, and
Cold Fusion - gauging from their popularity - and so I was just
wondering if similar tools for texis are forthcoming...

-John Koch - - - __o
Knowledge Systems, Inc. - - - - _ \<,_
<John.ksi@webplus.net> - - (_)/ (_)
(A NET-FRIENDLY SIG. http://www.ncsa.uiuc.edu/Edu/ICG/pt1.ch2.Etiquette.html )

[Thunderstone]

Hopefully you aren't running Netscape, gr_osview and Webinator on the same
machine. Because if your are, you're not measuring Webinator's performance,
but rather Netscape's CPU drain as it diplays the page.

I indexed the IRS's web site with my SGI Indy and ran a load
test program to check your findings. The test sequentially posts
100 random queries picked from a list of 1000. It took 14 seconds
to execute 100 queries.

Then I ran 10 of the load tests in parallel. It took 97 seconds to
complete the 1000 queries 10 at a time. At about 10 queries per second,
its not great, but its not bad either. Especially considering that
my Indy has a very slow narrow SCSI drive and I was running 4 copies of
netscape and two dozen other programs (including gr_osview, top, and osview ).

CPU utilization remained at about 30% for the duration of my tests,
but Top was indicating a load average of 4.6 . Context switches from
I/O waits were the primary culprit here I believe.

A side note: gr_osview by default has a very fast 'attack' time and
a really slow decay. It's misleading to use it in conjunction with
single short burst transactions like those done by HTTP. The much
uglier but more informative tty osview generally provides a much
better picture of a machine.


In identical applications Texis/Vortex consistently outperforms
the offerings from Oracle, Informix, PLS, Verity, Excalibur, Allaire,
and Netscape. How do we know this? Because we've replaced millions
of dollars worth of their stuff at many different high-visibility sites
across the net. Call us and we'll give you names and contacts to validate
this claim. And certainly no incarnation of a JavaScript is going to
match the performance levels of Vortex.


With respect to Graphical AppDev tools, we've done a lot of investigation
into this and are _very_ interested in providing one. The trouble that
we find ourselves with is that once you go beyond the most simplistic
applications, the efficacy of most of these tools falls apart.
We want a tool that can make non-trivial applications really trivial
to develop, and we just have not found the right model to work from.

Most of the large sites using Texis use one of the really good page layout
tools to create a mock-up of the HTML that will become the application.
Then, a programmer fills in the Vortex code around the prototype to
make it active. Admittedly this is not as much fun as drag and drop
widgets, but its very effective.

We're still looking for a better way.

Thanks,

Thunderstone

[Thunderstone]

Thunderstone kindly replies to my query:


We've indexed the www.hq.nasa.gov site (9000+ pages) and we'd search for,
say "tr*" (a non-sensical search, but again, we're just trying to
do some stress-testing). From the IRIX shell, "top -i2" often shows
70% - sometimes 85% - of the CPU being used by the process running "texis"
while at the same time, (tho' admittedly it can be hard to tell) my browser
was receiving a page of 10 of the matches. I'm assuming that when the
browser starts receiving data, the database search is complete and all
that remains is the presentation of the results.

Then I asked:

and Thunderstone replies:

Oh I believe you. But I'm trying to look NOT at the performance of database
searches but performance of the presenatation of the results of those searches.
As I mentioned, it was clear from my own testing that database searches
do indeed seem to be fastest when using vortex. So for large to humongous
databases - which I'm assuming you're referring to above - the impressive
performance of a vortex database mostly certainly would outweigh any issues
related to the performance of the execution of texis scripts. But for
small database, I'm thinking (tho' I'm willing to stand corrected) the
reverse might be true since the database search would be fairly quick under
most any database platform leaving the scripting used to present the search
results a proportionally bigger issue.

So my question has to do with comparing compiled JavaScript to texis -
not Informix/Oracle/PLS to vortex....?

On a closely related subject: I take it there are no internal meters?

-John Koch - - - __o
Knowledge Systems, Inc. - - - - _ \<,_
<John.ksi@webplus.net> - - (_)/ (_)
(A NET-FRIENDLY SIG. http://www.ncsa.uiuc.edu/Edu/ICG/pt1.ch2.Etiquette.html )

[Thunderstone]

That search _will_ eat a machine. Thats why other engines don't
even allow your to do that kind of wildcard search. If your're concerned
about user's killing your box with queries like that, use <sandr> to
remove any wildcard searches under four characters.

eg: <sandr ">>[^ \*]{1,3}\*" "" $query>

VORTEX IS TEXIS-WEB SCRIPT! It's not slow. Its only about 5% slower
than a hand coded 'C' program that does the same application.
We know this because the OLD Webinator was coded in C and the new
one written in Texis Web Script/Vortex is not discernably different.

Now you might want to get into a long winded discussion about
Texis compiled/linked into the web server here, and we DO do this too.
But Webinator must be a CGI because by the numbers, most of its userbase
does not have the resources/authority to re-compile their servers,
and we don't want the tech-support nightmare that's affiliated with
this either. (especially on the free version)