Page 1 of 1

GW Fails: 100 Redirect loop fetching (original) -- help!

Posted: Thu Mar 29, 2007 11:00 am
by enrique.delgado
Running "Webinator WWW Site Indexer Version 2.56 (Commercial)".

The re-walks have been failing for me with this error:

./gw -msd.cfg -v5 http://www.sirsidynix.com
Loading from lastrun
175 Table hosts not found in data dictionary
115 No such table: hosts in the database: /s/sirsi/WWW/searchdb/sd/
000 SQLExecute() failed with -1
Using meta data field
Getting http://129.41.240.45/robots.txt...Not there...Ok.
Adding todo: http://www.sirsidynix.com/
Saving options and URLs to lastrun
http://www.sirsidynix.com/
0: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0 Retrieving
0: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0
100 Redirect loop fetching (original) http://www.sirsidynix.com/
0: TotLinks: 0, Links: 0/ 0, Good: 0, New: 0
Visited 1 pages
Visited 0 pages
Visited 1 pages total
Indexing new pages
Using meta data field
set delexp=0;
set addexp='\alnum{2,30}';
create metamorph inverted index xhtmlbod on html(Title\Meta\Body);

I think this line is the problem:

100 Redirect loop fetching (original)

I tried everything I know of (wiping the db doing the initial populating, tryin to re-walk, etc) to no avail. My config file looks like this:

#Configuration file for indexing http://www.sirsidynix.com
#Command: ./gw -msd.cfg http://www.sirsidynix.com
#
#handle PDF files with the purchased plugin
napplication/pdf,pdf,/s/sirsi/WWW/htdocs/webinator/bin/anytotx
#
#allow big files so PDF doesn't get truncated
z100000000
#
# set the index page to index.php
Iindex.php
#
# allow php extensions:
fphp
#
#set the database to be used:
d/s/sirsi/WWW/searchdb/sd
#
#don't include ALT text in database
N

Can anybody help please?

GW Fails: 100 Redirect loop fetching (original) -- help!

Posted: Thu Mar 29, 2007 11:48 am
by mark
That page requires cookies and keeps redirecting you to the index.php page trying to give one. gw doesn't support cookies. You need a newer version of webinator (4 or later) with the web based admin interface.

GW Fails: 100 Redirect loop fetching (original) -- help!

Posted: Thu Mar 29, 2007 12:04 pm
by enrique.delgado
Thanks for the tip! I'll see if I can fix this. Thanks again :)

GW Fails: 100 Redirect loop fetching (original) -- help!

Posted: Thu Mar 29, 2007 3:17 pm
by enrique.delgado
Hi Mark, thanks for your help. I made it so that the site degrades gracefully if you don't support cookies or cookies are disabled. Hence, webinator is able to crawl the site once again.

Thanks!