date based fetching (was Quick question about a command)

Post Reply
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

date based fetching (was Quick question about a command)

Post by Thunderstone »



-e uses the "visited" time which is stored in the database. Not the modified
time that the server might provide. So all times are referring to the time
the page was last fetched.

-e refetches older than the specified date.
If today is the 23rd, "-3 days" means the 20th. So -e"-3 days" will fetch
all pages fetched before the 20th. "+3 days" would be the 26th. All pages
will have been fetched before then since it's in the future!

The only way to get "modified" pages is to ask the server about every page.
You can use the -V option to do "if modified since" queries and only
download modified pages. You can also use the -X option to delete pages
that no longer exist. (Both of these options only apply when using -e).

gw -e"-1 day" -X -V




Post Reply