Remove common navigation not working

john.santangelo · Post by **john.santangelo** » Thu Jan 10, 2008 8:52 am

How can I get some page navigation to stop showing up in the Abstract? I have abstract set to Query. I also have Remove Common set to Y.

Search for "fee schedule", a result has this abstract for this page:

http://www.floridamedicare.com/Part_A/Fee_Schedules/

TrialsComment SummariesRetired LCDsStatuses Find LCDs Reconsiderations Download Adobe Reader Fee Schedules New 2008 Payment Rates for Services Paid Under Medicare

This is common and is not limited to only this search query.

By the way, Thunderstone is not yet set up on that site externally so you cannot duplicate the results.

Post by **mark** » Thu Jan 10, 2008 10:32 am

Remove comon has limited usefulness with refresh crawls and when the boilerplate has dynamic bits. When possible it's better to use Keep Tags and/or Ignore Tags. For pages like that one try a Keep Tag Begin of
<div id="innerbody">
and a Keep Tag End of
<div id="footer">

john.santangelo · Post by **john.santangelo** » Thu Jan 10, 2008 1:36 pm

Well, I still get bad Abstracts, but for a different reason.

I have abstracts like:

<div class="meps_right">November 27, 2007</div><h2 class="meps_header">Latest NPI News Relative to SSNs, Legacy IDs and Much More</h2>

How would I set it up to not show HTML/CSS but just the text?

Post by **mark** » Thu Jan 10, 2008 3:04 pm

Sorry. Also add Keep tags Begin
<head>
and End
</head>

john.santangelo · Post by **john.santangelo** » Thu Jan 10, 2008 4:59 pm

Still have same issue. Do I need to identify every tag that I don't want to show?

I don't understand why, all of the sudden, html shows up in the results.

Post by **mark** » Thu Jan 10, 2008 5:23 pm

No. Make sure you put these items in "Keep Tags", not "Ignore Tags" and that <head> and </head> are on a line separate from the div entries. Without keeping the <head></head> info it forgets that it's an html document and treats it as plain text.

Make sure you're doing a new walk, not refresh, to ensure that the page is being refetched each time while you're testing.

john.santangelo · Post by **john.santangelo** » Fri Jan 11, 2008 9:08 am

Still having problem. Going to email help@thunderstone with screen grabs of all my settings to confirm.

Post by **mark** » Fri Jan 11, 2008 10:18 am

That should be support@thunderstone.com. Please send html saves, not screen pictures.