extensions

KMandalia · Post by **KMandalia** » Mon Sep 20, 2004 2:20 pm

does case matter? Do we have to specify .HTM and .htm suspecting some of the websites may use .HTM?

What is the effect of ignore case?

Is .pdf same as .PDF? Because it doesn't seem to index pages with .PDF (The site I am crawling has all pdf extensions as .PDF)

How to make it ignore case when considering webpages?

Post by **John** » Mon Sep 20, 2004 2:45 pm

By default it will ignore the case of the extension. You could change that in the dowalk script by editing the <$SSc_respectextcase=N> line.

KMandalia · Post by **KMandalia** » Mon Sep 20, 2004 2:57 pm

Thanks, but if you don't respect extension case then .PDF of .pdf shouldn't matter and it should walk .PDF when specified .pdf,right?

Post by **John** » Mon Sep 20, 2004 4:08 pm

Yes. What is the reason listed for the link not being followed?

KMandalia · Post by **KMandalia** » Mon Sep 20, 2004 4:23 pm

None. I started out with the base url and checked it's child. The one below and similar ones are not even listed there (unwanted extnsion should be there, I have respectcase=N)

http://www.cujournal.com/CUJCLASS.PDF

Post by **John** » Mon Sep 20, 2004 4:44 pm

Which page has the link to CUJCLASS.PDF?