Query log fields

Post Reply
legedza.henry
Posts: 142
Joined: Wed Jul 24, 2002 11:52 pm

Query log fields

Post by legedza.henry »

Hi there,

Just wondering, what do these fields mean in the excel query log

What is the difference between Query and Click and what do the URL's indicate in the URL field?

A large number of the urls listed next to Click are from msn search, googlebot etc. And the resultant URLs are completely unrelated to the original searchs.

eg
65.55.106.131 Query mobile phones 346
65.55.106.110 Click mobile phones http:/www.decs.sa.gov.au/sport/files/links/Rugby_Leagu_5.pdf
65.55.208.236 Click mobile phones http:/www.decs.sa.gov.au/speced2/files/links/Referral_Form_CYH_Delegate.doc
65.55.208.236 Click mobile phones http:/www.decs.sa.gov.au/sport/default.asp?id=29692&navgrp=2806
User avatar
jason112
Site Admin
Posts: 347
Joined: Tue Oct 26, 2004 5:35 pm

Query log fields

Post by jason112 »

> What is the difference between Query and Click and
> what do the URL's indicate in the URL field?

"Query" indicates that line shows a search that was performed. "Click" means that line shows a result that was clicked from a search.

The snippet you posted indicated that a search for "mobile phone" was performed and returned 346 results, and the next 3 lines are URLs that were followed from some search.

You can keep well behaved internet crawlers (google, msn, yahoo, etc) away from your search results by adding the following custom header to your search results:

<meta name="robots" content="noindex,nofollow">

> And the resultant URLs are completely unrelated to
> the original searchs.

I don't see how they're unrelated, the URLs you posted have the words "mobile" and "phone" in them, so they'll be returned as results.
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Query log fields

Post by mark »

And/or use robots.txt to keep the bots out of your texis app entirely.
legedza.henry
Posts: 142
Joined: Wed Jul 24, 2002 11:52 pm

Query log fields

Post by legedza.henry »

Thanks. So to confirm, I add the meta tag to the search script?

Where would I put the robots.txt file to keep bots out of the app competely?
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Query log fields

Post by mark »

legedza.henry
Posts: 142
Joined: Wed Jul 24, 2002 11:52 pm

Query log fields

Post by legedza.henry »

Hi there,

I added the metatag to the top.html but still getting bot entries:

6/10/2009 11:43 65.55.107.179 Query mobile phones
6/10/2009 11:51 66.249.71.148 Query mobile phones
6/10/2009 14:16 65.55.208.236 Query mobile phones
6/10/2009 14:28 65.55.208.232 Click mobile phones
6/10/2009 17:19 66.249.71.148 Query mobile phones
6/10/2009 17:21 66.249.71.148 Query mobile phones
6/10/2009 19:27 65.55.208.236 Query mobile phones
6/10/2009 19:38 65.55.106.139 Click mobile phones
6/10/2009 19:45 65.55.207.45 Query mobile phones
6/10/2009 21:08 65.55.207.122 Click mobile phones
6/10/2009 21:59 65.55.106.115 Click mobile phones
6/10/2009 22:38 65.55.207.96 Query mobile phones
6/10/2009 22:47 65.55.106.184 Click mobile phones
7/10/2009 1:53 65.55.208.241 Query mobile phones
7/10/2009 3:54 65.55.207.49 Click mobile phones
7/10/2009 4:46 65.55.106.211 Query mobile phones
7/10/2009 8:44 65.55.106.232 Query mobile phones
7/10/2009 9:15 65.55.207.51 Click mobile phones
7/10/2009 9:48 65.55.208.237 Query mobile phones
7/10/2009 10:07 65.55.106.185 Click mobile phones
7/10/2009 10:27 65.55.208.242 Query mobile phones
7/10/2009 11:24 65.55.208.244 Query mobile phones
7/10/2009 12:28 65.55.207.77 Click mobile phones

This IP is the other common one that appears: 66.249.71.148

if you goto to www.decs.sa.gov.au the search is visible there
User avatar
John
Site Admin
Posts: 2595
Joined: Mon Apr 24, 2000 3:18 pm
Location: Cleveland, OH
Contact:

Query log fields

Post by John »

The meta tag will stop the robot following the search results or more like this, but can't stop it fetching the page, as it doesn't see the meta-tag until the page has been fetched. The robots.txt would stop that.
John Turnbull
Thunderstone Software
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

Query log fields

Post by mark »

Also keep in mind that all this robot control stuff is really just suggestions. There's no enforcement. But the major bots are generally respectful, though it may take some time before robots.txt if checked again by any given bot.
Post Reply