Thunderstone Support Forums

This page: http://www.rockwellautomation.com/process/ has a robots meta tag set to noindex. I have set the crawl to obey Meta Robots:
Robots robots.txt: Y Meta: Y Placeholder: Y

This is the entry in List url's:
URL: http://www.rockwellautomation.com/process/
Depth: 1 click away from Home
Size: 456 bytes download, 0 bytes text
Indexed: 2008-07-17 01:04:52
Modified: 2008-07-17 01:04:52
Last Visit: 2008-07-17 01:04:52
Next Visit: 2008-07-24 01:04:52 Update Soon
Download time: Less than 1 second
Hash: 487ee1046f
Links: Parents, Children
Error: meta robots NOINDEX
Categories: -None-
Title: -None-
Description: -None-
Keywords: -None-
Meta data: -None-
Text Charset: -None-
Source MIME Type: -None-

But in Searches this page still shows up:
http://search2.rockwellautomation.com/t ... ry=process

Appliance scripts versions
Version: Search Appliance Server Version 5.01.1196094852 20071126 (i686-unknown-linux2.4.9-64-32)

Scripts Version: 6.3.1


Ideas?

They're showing up because you included URL in your search fields and have Placeholder set to Y. Turn off Placeholder to prevent them from going into the database at all. Placeholder keeps the url and it's info (but no text) in the database to aid in error reporting and the "show parents" of the default search results.