html in search results

rgwin0 · Post by **rgwin0** » Fri May 04, 2007 2:02 pm

I am getting unwanted html tags displayed in my search results. For example, note the div tags in the first couple hits here:

http://search.dvcotechnology.com/cgi-bi ... h_spherion

You can see I've modified the overall search display, so perhaps this is something I screwed up. But all I really did was remove some of the extra formatting and links from the results page. I didn't knowingly alter the actual result strings, and I'm pretty lost figuring out where to track this down. Any suggestions?

thanks,
rob

Post by **mark** » Fri May 04, 2007 2:46 pm

Looks like you're somehow displaying original html in the abstract instead of the extracted text. Did you change wha t the walk stores in the Body field?

Try an unmodified search script (call it something else so it doesn't collide with your live one). How does it look there?

rgwin0 · Post by **rgwin0** » Fri May 04, 2007 5:02 pm

I don't recall altering the Body field, where would I check that?

Testing with the original unmodified search script yields the same thing:

http://search.dvcotechnology.com/cgi-bi ... h_spherion

Are the html tags supposed to be stripped out on data storage or display?

rgwin0 · Post by **rgwin0** » Fri May 04, 2007 5:51 pm

Okay, so I'm fairly certain I haven't modified the crawling scripts, I wouldn't even know how to do that. So any idea how I can track down the problem?

Post by **mark** » Fri May 04, 2007 6:07 pm

It appears somehow related to using "Keep tags". Not sure exactly what's happening yet though...

rgwin0 · Post by **rgwin0** » Fri May 04, 2007 7:01 pm

My Keep Tags are "search-start" and "search-end", each surrounded in html comment tags. Viewing the source of the first hit from my example, I find the search-start tag, two div tags that aren't included in the results, two that are included, and then my search string. This makes me wonder if the need to have n characters of text preceding the search string is somehow overriding the need to strip html? Possible??

(opening tag brackets removed so as to not confuse the message board)

search-start-->
div id="wd_printable_content">
div class="wd_newsfeed_releases-detail">
div class="wd_news_releases-detail">
div class="detail_header">Spherion Survey

rgwin0 · Post by **rgwin0** » Mon May 07, 2007 1:20 pm

Yeah. Okay looks like my worries about posting html tags here were unfounded. So, my keep tags are:




The html of the original page looks like:


<div id="wd_printable_content">
<div class="wd_newsfeed_releases-detail">
<div class="wd_news_releases-detail">
<div class="detail_header">Spherion Survey: Industrial, Manufacturing

...and a search for "survey industrial" returns:

<div class="wd_news_releases-detail"> <div class="detail_header">Spherion
Survey: Industrial, Manufacturing...

Bizarre, huh?

rgwin0 · Post by **rgwin0** » Mon May 07, 2007 1:22 pm

I just realized I still wasn't too clear: The keep tags are indeed entered with the surrounding tags in the walk settings.

rgwin0 · Post by **rgwin0** » Mon May 07, 2007 1:46 pm

The original URL is the first result on my example results page noted above:

http://spherion.mediaroom.com/pressroom ... 3&item=360

rgwin0 · Post by **rgwin0** » Mon May 07, 2007 1:51 pm

And since you mentioned possible charset issues, here's my settings for that:

The server outputs this HTTP header:
Content-Type: text/html; charset=UTF-8

In webinator admin,
Storage Charset is blank.
Source Default Charset is "UTF-8"
XML UTF-8 is "N" (required to fix another known issue)