Meta Robots noindex is not working. pages are showing up in searches

Post Reply
jgdoke
Posts: 167
Joined: Wed Jul 14, 2004 10:52 am

Meta Robots noindex is not working. pages are showing up in searches

Post by jgdoke »

This page: http://www.rockwellautomation.com/process/ has a robots meta tag set to noindex. I have set the crawl to obey Meta Robots:
Robots robots.txt: Y Meta: Y Placeholder: Y

This is the entry in List url's:
URL: http://www.rockwellautomation.com/process/
Depth: 1 click away from Home
Size: 456 bytes download, 0 bytes text
Indexed: 2008-07-17 01:04:52
Modified: 2008-07-17 01:04:52
Last Visit: 2008-07-17 01:04:52
Next Visit: 2008-07-24 01:04:52 Update Soon
Download time: Less than 1 second
Hash: 487ee1046f
Links: Parents, Children
Error: meta robots NOINDEX
Categories: -None-
Title: -None-
Description: -None-
Keywords: -None-
Meta data: -None-
Text Charset: -None-
Source MIME Type: -None-

But in Searches this page still shows up:
http://search2.rockwellautomation.com/t ... ry=process

Appliance scripts versions
Version: Search Appliance Server Version 5.01.1196094852 20071126 (i686-unknown-linux2.4.9-64-32)

Scripts Version: 6.3.1


Ideas?
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Meta Robots noindex is not working. pages are showing up in searches

Post by mark »

They're showing up because you included URL in your search fields and have Placeholder set to Y. Turn off Placeholder to prevent them from going into the database at all. Placeholder keeps the url and it's info (but no text) in the database to aid in error reporting and the "show parents" of the default search results.
Post Reply