Thunderstone Support Forums

Hi, I am trying to <fetch> certain URL that I know is supposed to redirect to another page, but even then I was expecting <fetch> will return the html of the final page... which is not happening.

this is the sample code:
<urlcp maxredirs 10>
<fetch $thisurl>

here is the actual URL:
http://everquest.allakhazam.com/cgi-bin ... 1&save=yes


<fetch> produces this error:

<!-- 100 [walkeverquest]:87: *No Location header given in redirect* from http://everquest.allakhazam.com/cgi-bin ... 1&save=yes -->

plus :
*Bad redirect*
as return of <urlinfo "errmsg">


Please help. Is there anything I can set before or after <fetch> to avoid this situation?

Thank You

It looks as if that page merely redirects back to the referrer, and so without a referrer it doesn't do a proper redirect. There wouldn't be any point in crawling it anyway, as you could set a referrer, but you may as well fetch it directly.

that's a good point, I think REFERER is exactly what I need. But I still need instructions how to do it. I know that when I program on the server side, I can set the value of HTTP_REFERER env. variable. How do I set a referer header when programing a client (crawler)?

More details:
In fact there is a referring page, one example would be:
http://everquest.allakhazam.com/forum.html?forum=22

When I <fetch> this page it gives me back the HTML with DEFAULT filtering and layout options. I need to fetch it with different options. By studying the returned HTML, I discovered the FORM that does that. It starts with:

<form name="pref" action="/cgi-bin/prefs-new.pl">
...etc...

That's why I need to fetch this URL (the one with "prefs-new.pl"). I was hoping that the final return will be the redirection to the original (referer) page but with my custom filtering and layout options. Although, note that I am still a beginner in vortex, so I might be totally on the wrong track.

Thanks a lot!

OK, what you probably want is

<urlcp header Referer "someurl">


before the fetch.

this works!
thank you so much, especially for such a quick response.