Indexing public_html directories

User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Indexing public_html directories

Post by Thunderstone »



Hi,

We are using Webinator to search our site and would like to be able to add our users public_html directories to the database. Is there a way to do this? There directories are on a seperate server then the one gw is running on. Your help is greatly appreciated.

Thank You in Advance,
Mitch
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Indexing public_html directories

Post by Thunderstone »

User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Indexing public_html directories

Post by Thunderstone »



Is there a way to possibly feed gw the page where the links to these other pages are and let it index it from there?



--
Mitchell J. Mellin, Webmaster
Monmouth University
431 Cedar Ave
West Long Branch, NJ 07764
(908)571-3539
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Indexing public_html directories

Post by Thunderstone »



If the page with links to the other pages is on the same server as the user
pages then this will work, just give it that URL. Gw will confine itself
to the server it starts on by default.

Webmaster said:

User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Indexing public_html directories

Post by Thunderstone »



I just attempted indexing the users directories from within the page which contains the links. It did not work gw only indexed the pages that were within the main server root. I am using version 1.3 for IRIX and the page is within frames could either one of these two things be what is causing the problem.

Thank You,
Mitch



User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Indexing public_html directories

Post by Thunderstone »



Are the links on the page directory links? If so, do they end with a
trailing slash (/). It is common, but incorrect to leave off the trailing
slash on directory names. Your web browser will go after any old thing
you type, and the webserver will often correct the mistake for you.
gw will not attempt to go after it unless it has a valid extension or
a trailing slash.