inclusion logic
Posted: Mon Jun 28, 2004 7:52 am
Hi,
I am trying to crawl a site based on words in the URL regardless of position, for example, the word 'health' and exclude everything else.
eg.: I start at something.org and in it are 5 links:
1) /subdir1/health
2) /subdir2/beauty
3) /health
4) /subdir3/subdir4/health.asp
5) /a/b/c/d/e/f/healthy/
I only want to include 1 and 3 and all their children with the word 'health' in it.
I used the inclusion REX field and so far everything is crawled even when the query word is not in it.
I don't want to include anything else or use categories.
Is there any way to this?
I am trying to crawl a site based on words in the URL regardless of position, for example, the word 'health' and exclude everything else.
eg.: I start at something.org and in it are 5 links:
1) /subdir1/health
2) /subdir2/beauty
3) /health
4) /subdir3/subdir4/health.asp
5) /a/b/c/d/e/f/healthy/
I only want to include 1 and 3 and all their children with the word 'health' in it.
I used the inclusion REX field and so far everything is crawled even when the query word is not in it.
I don't want to include anything else or use categories.
Is there any way to this?