Page 1 of 1

pages not indexed

Posted: Fri Mar 15, 2002 6:28 pm
by anixyz
Some of the html pages from our site are not being indexed and so not coming up in search results.
What are all the possibilities that gw does not index some html pages.(like pages navigated to via Javascript)

Some pages are not "present" in html table but are present in "refs" table.whats going on here....


pages not indexed

Posted: Fri Mar 15, 2002 11:35 pm
by mark
There are *many* option settings that control what pages are indexed or not. See the manual for the long list of options and what each one does.

The refs table contains all of the urls seen on the fetched pages whether they were rejected for walking or not.

If you have specific questions about particular urls in conjunction with your particular option settings let us know.