Will not crawl my page

robert.phillips · Post by **robert.phillips** » Fri Mar 25, 2005 11:29 am

i'm trying to craw the following data. i setup a new profile, the base URL is this:

http://leagis/documentlibrarylinks/webform2.aspx

the html in this file is below. the walk status never shows anything beyond the basic page.

<HTML>
<HEAD>
<title>WebForm2</title>
</HEAD>
<body>
<form name="Form1" method="post" action="webform2.aspx" id="Form1">
<div id="dvLinks" style="WIDTH: 100%">
<a href="webform2.aspx?session=2005&id=38&docType=Bills">2005 Bills</a> | <a href="webform2.aspx?session=2005&id=38&docType=Amendments">2005 Amendments</a> | <a href="webform2.aspx?session=2005&id=38&docType=Analysis">2005 Analysis</a> | <br>
<a href="webform2.aspx?session=2004O&id=39&docType=Bills">2004O Bills</a> | <a href="webform2.aspx?session=2004O&id=39&docType=Amendments">2004O Amendments</a> | <a href="webform2.aspx?session=2004O&id=39&docType=Analysis">2004O Analysis</a> | <br>
<a href="webform2.aspx?session=2004A&id=41&docType=Bills">2004A Bills</a> | <a href="webform2.aspx?session=2004A&id=41&docType=Amendments">2004A Amendments</a> | <a href="webform2.aspx?session=2004A&id=41&docType=Analysis">2004A Analysis</a> | <br>
<a href="webform2.aspx?session=2004&id=36&docType=Bills">2004 Bills</a> | <a href="webform2.aspx?session=2004&id=36&docType=Amendments">2004 Amendments</a> | <a href="webform2.aspx?session=2004&id=36&docType=Analysis">2004 Analysis</a> | <br>
<a href="webform2.aspx?session=2003E&id=37&docType=Bills">2003E Bills</a> | <a href="webform2.aspx?session=2003E&id=37&docType=Amendments">2003E Amendments</a> | <a href="webform2.aspx?session=2003E&id=37&docType=Analysis">2003E Analysis</a> | <br>
<a href="webform2.aspx?session=2003D&id=35&docType=Bills">2003D Bills</a> | <a href="webform2.aspx?session=2003D&id=35&docType=Amendments">2003D Amendments</a> | <a href="webform2.aspx?session=2003D&id=35&docType=Analysis">2003D Analysis</a> | <br>
<a href="webform2.aspx?session=2003C&id=34&docType=Bills">2003C Bills</a> | <a href="webform2.aspx?session=2003C&id=34&docType=Amendments">2003C Amendments</a> | <a href="webform2.aspx?session=2003C&id=34&docType=Analysis">2003C Analysis</a> | <br>
<a href="webform2.aspx?session=2003B&id=33&docType=Bills">2003B Bills</a> | <a href="webform2.aspx?session=2003B&id=33&docType=Amendments">2003B Amendments</a> | <a href="webform2.aspx?session=2003B&id=33&docType=Analysis">2003B Analysis</a> | <br>
<a href="webform2.aspx?session=2003A&id=32&docType=Bills">2003A Bills</a> | <a href="webform2.aspx?session=2003A&id=32&docType=Amendments">2003A Amendments</a> | <a href="webform2.aspx?session=2003A&id=32&docType=Analysis">2003A Analysis</a> | <br>
<a href="webform2.aspx?session=2003&id=28&docType=Bills">2003 Bills</a> | <a href="webform2.aspx?session=2003&id=28&docType=Amendments">2003 Amendments</a> | <a href="webform2.aspx?session=2003&id=28&docType=Analysis">2003 Analysis</a> | <br>
<a href="webform2.aspx?session=2002O&id=29&docType=Bills">2002O Bills</a> | <a href="webform2.aspx?session=2002O&id=29&docType=Amendments">2002O Amendments</a> | <a href="webform2.aspx?session=2002O&id=29&docType=Analysis">2002O Analysis</a> | <br>
<a href="webform2.aspx?session=2002E&id=16&docType=Bills">2002E Bills</a> | <a href="webform2.aspx?session=2002E&id=16&docType=Amendments">2002E Amendments</a> | <a href="webform2.aspx?session=2002E&id=16&docType=Analysis">2002E Analysis</a> | <br>
<a href="webform2.aspx?session=2002D&id=15&docType=Bills">2002D Bills</a> | <a href="webform2.aspx?session=2002D&id=15&docType=Amendments">2002D Amendments</a> | <a href="webform2.aspx?session=2002D&id=15&docType=Analysis">2002D Analysis</a> | <br>
<a href="webform2.aspx?session=2002&id=4&docType=Bills">2002 Bills</a> | <a href="webform2.aspx?session=2002&id=4&docType=Amendments">2002 Amendments</a> | <a href="webform2.aspx?session=2002&id=4&docType=Analysis">2002 Analysis</a> | <br>
<a href="webform2.aspx?session=2001C&id=8&docType=Bills">2001C Bills</a> | <a href="webform2.aspx?session=2001C&id=8&docType=Amendments">2001C Amendments</a> | <a href="webform2.aspx?session=2001C&id=8&docType=Analysis">2001C Analysis</a> | <br>
<a href="webform2.aspx?session=2001B&id=5&docType=Bills">2001B Bills</a> | <a href="webform2.aspx?session=2001B&id=5&docType=Amendments">2001B Amendments</a> | <a href="webform2.aspx?session=2001B&id=5&docType=Analysis">2001B Analysis</a> | <br>
</div>
</form>
</body>
</HTML>

Post by **John** » Fri Mar 25, 2005 11:50 am

Make sure you have .aspx as an extension to walk, "?" out of the exclusions and "Strip Queries" set to "N".

robert.phillips · Post by **robert.phillips** » Fri Mar 25, 2005 1:21 pm

That worked, thanks.