aspell making a mess with suggestions for noise words

Post Reply
rjshelq
Posts: 75
Joined: Thu Nov 17, 2005 3:25 pm

aspell making a mess with suggestions for noise words

Post by rjshelq »

With Webinator 6.10, there seem to be some oddities when a search phrase includes noise words. Here are a series of tests which depict the problem:

First, a search on my site for a non-existent word: Magdelin results in no hits and a correct suggestion Did you mean: magdelene.

So far so good. Next, a search for: Mary Magdelin also results in no hits and a correct suggestion Did you mean: mary magdelene.

But, a query which includes a couple of noise words such as: then are Mary Magdelin results in no hits and produces unhelpful suggestions such as Did you mean: ten re mary madeliene and Did you mean: ten care mary madeliene.

It appears that Aspell is receiving the noise words, which do not appear in its dictionary, so then Aspell spends all it's time trying to "correct" the correctly spelled noise words. Sometimes the results of letting Aspell process the noise words are quite humorous, but the nonsense can be quite confusing and not at all useful to the end-user.

It seems that either Aspell should not receive the correctly spelled noise words, or that the Aspell dictionary should include the noise words so that Aspell recognizes them as valid words.

Do you have solution for this situation?

Would it be reasonable to alter the script to avoid sending correctly spelled query noise words to Aspell?

Or, perhaps would it be better to include the noise words in the dictionary and then ignore any noise suggestions received from Aspell?

Or, what about checking individual word-by-word spelling (including noise words) before ever beginning a search?
rjshelq
Posts: 75
Joined: Thu Nov 17, 2005 3:25 pm

aspell making a mess with suggestions for noise words

Post by rjshelq »

Do you have any thoughts about this Aspell problem??
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

aspell making a mess with suggestions for noise words

Post by mark »

Don't have a great solution off the top...
Adding noise words to the dictionary would open up suggesting noise over good words in all suggestions but could usefully correct things like "hte" to "the".
Skipping over noise for suggestions could miss useful suggestions, especially if what appears to be a correctly spelled noise word is really a mispelled good word.
Your last suggestion is more or less much what happens now.
rjshelq
Posts: 75
Joined: Thu Nov 17, 2005 3:25 pm

aspell making a mess with suggestions for noise words

Post by rjshelq »

Is there an easy way that I can add the noise words back into the Aspell dictionary (at least to do some tests)?
User avatar
mark
Site Admin
Posts: 5513
Joined: Tue Apr 25, 2000 6:56 pm

aspell making a mess with suggestions for noise words

Post by mark »

Comment out the line
<xtree insert $noiseWords "aspellnoise">
and remake the dictionary (see updateindex entry point).
Post Reply