Does setting the following can make refreshing urls almost identical to newly discovered urls?
min refresh=1 min
max refresh=1 day
default refresh=1 hr
nextcheck=asap
I have a folder on our website that I want to be fetched first every time I start a refresh. Since refresh is "calculated" and since I want to crawl this folder when something new gets added (but other things in the folder don't change at all), I wanted to override refresh timings for everything within that folder.
Also, say I want to refresh everything in database but don't want to go through the trouble of backing up querylog (the only reason i don't want to do a new walk), can I,
1) delete everything in todo
2) and set nextcheck for everything to asap ('now')
is there any other thing that distinguishes new from refresh?
min refresh=1 min
max refresh=1 day
default refresh=1 hr
nextcheck=asap
I have a folder on our website that I want to be fetched first every time I start a refresh. Since refresh is "calculated" and since I want to crawl this folder when something new gets added (but other things in the folder don't change at all), I wanted to override refresh timings for everything within that folder.
Also, say I want to refresh everything in database but don't want to go through the trouble of backing up querylog (the only reason i don't want to do a new walk), can I,
1) delete everything in todo
2) and set nextcheck for everything to asap ('now')
is there any other thing that distinguishes new from refresh?