Query re robots tags

[legedza.henry]

Hi there,

We are hitting the limits of our current license (20,000) and are trying to find ways to reduce the number of pages being indexed.

An example of such a page is this:

http://www.sacereview.sa.gov.au/a8_publ ... avgrp=1003

When you view it you'll notice it is only a menu bar. Ecah item then links to specific content

It has a robots tag of noindex/Follow yet it is stored within the DB.

Is this what should be happening?
Is there anyway to avoid these pages being stored?

Thanks

[John]

Yes, you can change the dowalk script to not keep those by changing the line:

<$SSc_metarobotsplaceholder=Y> <!-- keep url for "noindex" pages -->

to set the variable to "N". That will lose the ability to walk through parent pages in the List/Edit urls.

[legedza.henry]

Thanks for the tip. My only other question is how does one find out about these particular functionsd of dowalk.

I've had a look through the docs and the website and can't find any mention of this function (metarobotsplaceholder). is there anywhere that I can find these things documented?

Thanks

[mark]

I think that may be the only internal option (not on admin interface). It's very rarely used and breaks the parent link surfing so it's not mentioned. Looking at the "defaults" function in dowalk can provide some insight.