Page 1 of 2

meta keywords not updated

Posted: Tue Nov 23, 2004 4:26 pm
by john208
We are using Webinator Professional 5.0.6. We just added some new meta keywords and performed a rewalk of the site. I did a select on the keywords column in the html table, and the old keywords are there. What do I need to do to update the keywords so a search will find those keywords.

Thanks, John

meta keywords not updated

Posted: Tue Nov 23, 2004 4:48 pm
by John
If you do a New walk instead of Refresh it will pull all the new content in.

meta keywords not updated

Posted: Wed Nov 24, 2004 10:31 pm
by john208
Is there any other changes that requires a new walk before the content is picked up? We have only the rewalked scheduled automatically because it is so much quicker than a new walk. Are there any other cases where we need to do a manual new walk?

Thanks, John

meta keywords not updated

Posted: Wed Nov 24, 2004 11:10 pm
by John
When a refresh is done the rewalk will look at the content that is scheduled for a current refresh, and not update the content where the Last-Modified time hasn't changed, or where the text on the page hasn't changed. Currently changes of meta information or title will not count as significant enough to update the index. We will be changing that. If you have updated the keywords on a significant number of pages then the new walk will be faster. If most of the content hasn't changed the refresh will be faster.

meta keywords not updated

Posted: Thu Nov 25, 2004 12:08 am
by john208
How does the Webinator know what is the last modified time? Currently it is on the same server as the website so I suppose it can look at the filesystem. But we are planning to move the Webinator to a dedicated server, at which point, it can only use an external ip address. Also, the site is an ASP site so the dynamic content can change without the ASP page being modified. Is it able to find changes in dynamic content?

Thanks, John

meta keywords not updated

Posted: Thu Nov 25, 2004 10:35 am
by John
The webserver sends a last modified time, and Webinator will request the page if it was modified since that time. That can save the bandwidth of downloading the page.

If the webserver does send the page, either because it is dynamic or the last modified time was changed Webinator will then compare the actual text on the page to see if it changed or not. We will be updating Webinator to also look at the meta information to determine if the page changed.

meta keywords not updated

Posted: Mon Nov 29, 2004 11:58 am
by john208
Hello,

I performed a new walk and now the page in question is no longer in the html table. The page is linked from the menu that is on every page. Why would the meta keywords not be picked up? The meta keywords for other pages are in the html table but I'm not sure if all pages are in the table.

Thanks, John

meta keywords not updated

Posted: Mon Nov 29, 2004 1:59 pm
by mark
Meta keywords are fairly irrelevant if the page isn't even being indexed. Is the page a javascript reference? Do you have javascript turned on? Did you get any errors for that page in the walk status?

Go to list/edit urls. Lookup a page that links to the missing page. Click it's "Children" link and see if the missing url is listed. Is it plain text or is it clickable? If it's clickable it's in the database. Is there an error message to the right of the url?

If the url's not in the Children list it wasn't discovered on the page. If it's listed it was discovered but not stored. If there's an error listed with it that's the problem. If not, one of the exclude rules probably skipped it. Turn verbosity up to 4 and do a new walk if you can't see why it was skipped.

meta keywords not updated

Posted: Mon Nov 29, 2004 3:14 pm
by john208
We have a calendar page which has links for the next month and previous month. The webinator is walking the years 1780 through 2530. I see a timeout error reading from the calendar page.

Could the Webinator have aborted without walking further pages? Can I assume that the Max Depth setting will limit the number of calendar month pages that it will traverse?

Thanks, John

meta keywords not updated

Posted: Mon Nov 29, 2004 4:00 pm
by mark
If there's a timeout on a page all links from that page will be lost. The walk will continue with whatever other links it still has to do though.

Yes, max depth will prevent it from walking the whole calendar.