Page 1 of 1

Indexing sites that contain asp redirects

Posted: Mon Jan 10, 2005 12:16 pm
by j.gresham
Hi

I was just wondering if Webinator has problems with navigating a site that uses ASP server side re-directs. I am trying to index a site that behaves in the following way.

1. On hitting the site the user is prompted to logon (The site is protected using windows security)

2. On hitting the index page the user is immediately redirected to a page called SessionSet.asp' (First Redirect) this page according to the users logon status populates various session variables for later use.

3. Once these variables have been populated 'SessionSet.asp' then redirects to the relevant index page for the subsection to which the user has access.

The problem I have it that Webinator refuses to crawl the site strangly it seems to think that the SessionSet.asp file is offsite. Setting the crawl to index offsite pages has no effect.

Any help would be appreciated.

Checking for broken hyperlinks...

The link : http://dtlr3/neighbourhood
Had this error: Error translating. Plugin disabled ( -fother)

The link : /SessionSet.ASP
Had this error: Offsite
Referenced by : http://dtlr3.lan.kudos.com/neighbourhood/

The link : http://dtlr3.lan.kudos.com/neighbourhood/
Had this error: Offsite
Referenced by : http://dtlr3.lan.kudos.com/neighbourhood

The link : http://dtlr3.lan.kudos.com/SessionSet.ASP
Had this error: Offsite
Referenced by : http://dtlr3.lan.kudos.com/neighbourhood/

The link : http://dtlr3/stylesheets/explorer_win.css
Had this error: Error translating. Plugin disabled ( -fother)
Referenced by : http://dtlr3/

Indexing sites that contain asp redirects

Posted: Mon Jan 10, 2005 3:41 pm
by mark
http://dtlr3 is a different site than http://dtlr3.lan.kudos.com because the hostname is different. Use the proper hostname in the base url.

To do windows authentication you need a recent version (within the past year if you have a windows Webinator or after october if you have *nix Webinator).

But if the login was windows based I wouldn't expect the asp app to be doing the propagation. Perhaps it's really form based and you need to use http-post in your base url
http://www.thunderstone.com/texis/site/ ... y=Base+URL

Indexing sites that contain asp redirects

Posted: Tue Jan 11, 2005 10:22 am
by j.gresham
Hi

I did get the indexer running be changing the value in the User Agent field. However on subsequent attempts to index the site I am now receiving the following error

d:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(doprimer) 265: HttpSendRequest() to dtlr3:80 failed: ERROR_12015_317 in the function htworktillwait

Any ideas what might be causing this.

Thanks in advance

Indexing sites that contain asp redirects

Posted: Tue Jan 11, 2005 10:58 am
by Kai
12015 is a login failure, which would probably indicate the user/password was incorrect. Verify that the user/pass you gave in Login Info under All Walk Settings is correct, and works from the machine that Webinator is crawling from.

Indexing sites that contain asp redirects

Posted: Tue Jan 11, 2005 11:25 am
by j.gresham
I've checked the username and password and they are both correct.

Indexing sites that contain asp redirects

Posted: Wed Jan 12, 2005 2:33 pm
by Kai
Are you giving a domain for the user (eg. Domain\User)? It may be that the default domain is different for the machine that Webinator is accessing from than normal logins elsewhere.

Indexing sites that contain asp redirects

Posted: Thu Jan 13, 2005 6:13 am
by j.gresham
Hi

I tried entering the logon as domain\username but I still get the same error.

Indexing sites that contain asp redirects

Posted: Fri Jan 14, 2005 2:02 pm
by Kai
Have you tried setting Net Mode to Internal? This should work with NTLM if you have Webinator 5.1+.