Webinator crawl order

rluan · Post by **rluan** » Tue Jul 24, 2001 2:57 pm

Thank you for your reply. My observation shows that the 'gw' engine does follow the ASCII order of numbers that are part of target Urls to crawl. For instance, if there are five links in one page listed as

1. www.somesite.com/some.jsp?somepara=1
2. www.somesite.com/some.jsp?somepara=8
3. www.somesite.com/some.jsp?somepara=11
4. www.somesite.com/some.jsp?somepara=13
5. www.somesite.com/some.jsp?somepara=7

The crawl order will be 1, 3, 4, 5,2, the same order as the ASCII order of the enclosed numbers.

Is my observation correct? Are there other ruls to define crawl orders? Thanks again for your help.

-----Original Message-----
From: Thunderstone Technical Support
Sent: Monday, July 23, 2001 1:05 PM
Subject: Re: Webinator crawl order within one page

There's no defined order. They may even be interleaved with links from
other pages.

Question:
> Suppose there are 100 links in a Web page named link1, link2, link3, link4,
> ...., link99, link100 appearing from the top to the bottom of this page.
> When Webinator 'gw's this page, does it pick up these links in a specific
> order, say, from link1, link2 and all the way up to link100?
>
> Thanks.
>
>

Post by **Kai** » Tue Jul 24, 2001 3:04 pm

That order happens to be a side effect of how gw processes links. But there is no guarantee that links will always be walked in ASCII or any other particular order.