Date page last modified

Post Reply
User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Date page last modified

Post by Thunderstone »



Hi,
I want to index several sites using webinator. After each weekly index I
want to query the database to find all the documents that have been modified
or added, to these distant sites, within the last week. Is the date a
document was last modified stored within the database? There seems to be no
clear answer to this in the manual.

What exactly does the field 'Visited' in table 'html' actually contain? Is
it the date is was actually retrieved? If so, with the -V option am I right
in thinking this date will not change if the document does not change?

Is there a full list of database fields. The manual claims to only list
"some of the interesting database fields"?

thanks,

Leonard
--
Dr Leonard Newnham tel: +44 (0)1923 664117
Centre for Construction IT fax: +44 (0)1923 664689
Building Research Establishment email: newnhaml@bre.co.uk
Garston www: http://www.bre.co.uk/ccit
Watford WD2 7JR UK




User avatar
Thunderstone
Site Admin
Posts: 2504
Joined: Wed Jun 07, 2000 6:20 pm

Date page last modified

Post by Thunderstone »




The only date in the database is the last fetched date in the Visited field.
It is updated every time the document is stored. Unmodified documents
will not be stored.

Go to the list archive at http://www.thunderstone.com/texis/webinator/listproc/
and search for
date based fetching

for a discussion of fetching only modified pages.


The manual also provides the SQL to list all of the fields.



Post Reply