Indexer not following links in form

Post Reply
greenwoodc
Posts: 4
Joined: Fri Jul 27, 2007 2:43 pm

Indexer not following links in form

Post by greenwoodc »

I'm reasonably certain that there is no setting that will make webinator follow the links in the forms on this page and the pages it goes to: http://www.edmonds.wednet.edu/technolog ... /index.htm

They aren't my pages, so redesigning them is possible but not the preferred solution.

The pages are all located in the same directory, so I would want webinator to walk the directory. I know that webinator needs links.

Does anyone already have a script I could use (we're a non-profit school district, so free is appreciated) that automates creating an HTML file with links to all files in a directory? Then I could just tell webinator to look at that page. I can imagine a script working, I just don't want to apend a week (I'm a newbie to scripting) recreating the wheel if someone already has one.

OR.....Does anyone have a better idea?

Thanks!
Cathi
User avatar
John
Site Admin
Posts: 2622
Joined: Mon Apr 24, 2000 3:18 pm
Location: Cleveland, OH
Contact:

Indexer not following links in form

Post by John »

There are a couple options you have there. Our JavaScript plugin should work in that case as it uses JavaScript for the links, although that is not a free option.

Another option would be to have Webinator extract the links from that page, and turn the option value= into an a href= which would require some scripting skills.

The advantages to those approaches is that you won't try and index pages that aren't linked in. In Vortex the code to generate the links would be something like:

<A NAME=listfiles PUBLIC>
<stat ROW MAXDEPTH=10 /path/to/replace_levy_parts>
http://www.edmonds.wednet.edu/technolog ... parts/$ret
</stat>
</A>
John Turnbull
Thunderstone Software
User avatar
mark
Site Admin
Posts: 5519
Joined: Tue Apr 25, 2000 6:56 pm

Indexer not following links in form

Post by mark »

There's no setting to do that directly. You could modify dowalk slightly to extract those links and others like them. In the function "<a name=getlinks" find <getscriptstrlinks> and insert the following code just before that line:

<rex '<option value\="\P=[^"]{1,100}>>\.htm=l?\F">' $htmlpage>
<urlutil abs $base $ret>
<$u = $u $ret>
User avatar
mark
Site Admin
Posts: 5519
Joined: Tue Apr 25, 2000 6:56 pm

Indexer not following links in form

Post by mark »

John, FYI, Javascript processing won't get those links. Even with scriptvaryevents turned on.
greenwoodc
Posts: 4
Joined: Fri Jul 27, 2007 2:43 pm

Indexer not following links in form

Post by greenwoodc »

Woo hoo! I went with Mark's suggestion and that worked beautifully! Thank you!
Post Reply