Indexing sites that contain asp redirects

j.gresham · Post by **j.gresham** » Mon Jan 10, 2005 12:16 pm

Hi

I was just wondering if Webinator has problems with navigating a site that uses ASP server side re-directs. I am trying to index a site that behaves in the following way.

1. On hitting the site the user is prompted to logon (The site is protected using windows security)

2. On hitting the index page the user is immediately redirected to a page called SessionSet.asp' (First Redirect) this page according to the users logon status populates various session variables for later use.

3. Once these variables have been populated 'SessionSet.asp' then redirects to the relevant index page for the subsection to which the user has access.

The problem I have it that Webinator refuses to crawl the site strangly it seems to think that the SessionSet.asp file is offsite. Setting the crawl to index offsite pages has no effect.

Any help would be appreciated.

Checking for broken hyperlinks...

The link : http://dtlr3/neighbourhood
Had this error: Error translating. Plugin disabled ( -fother)

The link : /SessionSet.ASP
Had this error: Offsite
Referenced by : http://dtlr3.lan.kudos.com/neighbourhood/

The link : http://dtlr3.lan.kudos.com/neighbourhood/
Had this error: Offsite
Referenced by : http://dtlr3.lan.kudos.com/neighbourhood

The link : http://dtlr3.lan.kudos.com/SessionSet.ASP
Had this error: Offsite
Referenced by : http://dtlr3.lan.kudos.com/neighbourhood/

The link : http://dtlr3/stylesheets/explorer_win.css
Had this error: Error translating. Plugin disabled ( -fother)
Referenced by : http://dtlr3/

Post by **mark** » Mon Jan 10, 2005 3:41 pm

http://dtlr3 is a different site than http://dtlr3.lan.kudos.com because the hostname is different. Use the proper hostname in the base url.

To do windows authentication you need a recent version (within the past year if you have a windows Webinator or after october if you have *nix Webinator).

But if the login was windows based I wouldn't expect the asp app to be doing the propagation. Perhaps it's really form based and you need to use http-post in your base url
http://www.thunderstone.com/texis/site/ ... y=Base+URL

j.gresham · Post by **j.gresham** » Tue Jan 11, 2005 10:22 am

Hi

I did get the indexer running be changing the value in the User Agent field. However on subsequent attempts to index the site I am now receiving the following error

d:\Program Files\Thunderstone Software\Webinator\texis\scripts/Webinator/dowalk(doprimer) 265: HttpSendRequest() to dtlr3:80 failed: ERROR_12015_317 in the function htworktillwait

Any ideas what might be causing this.

Thanks in advance

Post by **Kai** » Tue Jan 11, 2005 10:58 am

12015 is a login failure, which would probably indicate the user/password was incorrect. Verify that the user/pass you gave in Login Info under All Walk Settings is correct, and works from the machine that Webinator is crawling from.

j.gresham · Post by **j.gresham** » Tue Jan 11, 2005 11:25 am

I've checked the username and password and they are both correct.

Post by **Kai** » Wed Jan 12, 2005 2:33 pm

Are you giving a domain for the user (eg. Domain\User)? It may be that the default domain is different for the machine that Webinator is accessing from than normal logins elsewhere.

j.gresham · Post by **j.gresham** » Thu Jan 13, 2005 6:13 am

Hi

I tried entering the logon as domain\username but I still get the same error.

Post by **Kai** » Fri Jan 14, 2005 2:02 pm

Have you tried setting Net Mode to Internal? This should work with NTLM if you have Webinator 5.1+.