Filtering out unwanted URLs

[scott.hovey]

In our crawl, we want it to crawl pages for the URL's but don't want to include those particular pages. Are we missing a setting that allows us to do that?

[mark]

Use meta robots on those pages or see "exclude by field" under all walk settings. Use field "Url".