How do I execute a query with multibyte characters. Eg :
select * from person_profile where person_name='multibyte characters'
I have set the following expressions while indexing.
What else needs to be set for multibyte characters.
set delexp=0;
set addexp='[\alnum\x80-\xff]{1,70}';
set addexp='[\alnum\x80-\xff.]{1,70}>>[.''\-_]=[\alnum\x80-\xff.]{1,70}';
set addexp='\digit=[\digit\.,+]{1,30}';
set addexp='>>[\xc0-\xfd]=[\x80-\xbf]+';
set addexp='>>\alnum=[\alnum\x5c\x5f\x2d\x26\x23\x40\x2e\x25\x2c]{1,30}';
The index expressions look correct for a multibyte string, however your SQL is using an "=" instead of LIKE. Metamorph indexes are used for LIKE queries, standard indexes for equals. Also make sure the character set you used for storage matches the query character set.
You probably want UTF-8 for storage. Then you just have to ensure that the query coming from the browser is also UTF-8. There's no guarantee that it will be but chances are that if you're searching for Japanese it will be UTF-8. Here's one way to check the query:
<rex row "\?" $query></rex><$x=$loop> <!-- count initial `?' -->
<strfmt "%!hV" $query> <!-- attempt UTF-8 decode -->
<rex row "\?" $ret></rex> <!-- count `?' again -->
<if $loop gt $x> <!-- more means not UTF-8 -->
<strfmt "%V" $query><$query=$ret> <!-- convert to UTF-8 -->
</if>
The easiest is for the text to be stored in UTF-8. You already have the index expression to index UTF-8 characters. One issue may be the lack of word boundaries in the text. You can test that by doing a search with a space between each Japanese character. If that works then you can look at the Webinator search scripts which have functions to transform a query into a phrased query of the individual characters.