difference between page file and url file.

jgdoke · Post by **jgdoke** » Mon Mar 10, 2008 5:21 pm

What is the difference? What determines which one to use?

Post by **jason112** » Mon Mar 10, 2008 5:48 pm

The child links of pages listed in Page File are not crawled. The URLs listed in the file are crawled, and then nothing more (baring other BaseURLs & such).

Url File acts essentially as an external list of BaseURLs.

jgdoke · Post by **jgdoke** » Mon Mar 10, 2008 6:00 pm

Any way to have just the links in page file crawled and nothing else??

Post by **jason112** » Tue Mar 11, 2008 9:57 am

> Any way to have just the links in page file crawled and
> nothing else??

That's what page file does. I was saying that if you specify a Base URL _in addition_ to a page file then the base url will be crawled normally. But if you only specify a page file, then only the links listed in the page file will be crawled.