I have a strange problem while crawling the sites.When i started the crawling success and failure pages get increased .The sites get crawled and everything worked fine.But suddenly i noticed in the walk status of the webinator that success rate is decreasing.It get decreased from 1079 to 609 and same thing happened with the failed page.
the walk Status was somewhat like
47 pages in todo
0 pages scheduled to be refreshed in the next hour
9,209 pages visited in the last hour (519 success/8,690 failed)
3,695 pages in index
Really weird.....Can you tell the possible reason of this strange behavior?
Are you talking about the like of pages visited in the last hour? If so it is possible that you are crawling a slower site, or slower portion of the site, e.g. dynamically generated, and so the walk rate has decreased, and the number of successful and failed pages in the last hour has decreased.