unicode - MySQL treats ÅÄÖ as AAO?

Question

These two querys gives me the exact same result:

select * from topics where name='Harligt';
select * from topics where name='H?rligt';

How is this possible? Seems like mysql translates ??? to aao when it searches. Is there some way to turn this off?

I use utf-8 encoding everywhere as far as i know. The same problem occurs both from terminal and from php.

Answer 1

Yes, this is standard behaviour in the non-language-specific unicode collations.

9.1.13.1. Unicode Character Sets

To further illustrate, the following equalities hold in both utf8_general_ci and utf8_unicode_ci (for the effect this has in comparisons or when doing searches, see Section 9.1.7.7, “Examples of the Effect of Collation”):

? = A ? = O ü = U

See also Examples of the effect of collation

You need to either

• use a collation that doesn't have this "feature" (namely utf8_bin, but that has other consequences)

• use a different collation for the query only. This should work:

   select * from topics where name='Harligt' COLLATE utf8_bin;
  

it becomes more difficult if you want to do a case insensitive LIKE but not have the ? = A umlaut conversion. I know no mySQL collation that is case insensitive and does not do this kind of implicit umlaut conversion. If anybody knows one, I'd be interested to hear about it.

Related:

• Looking for case insensitive MySQL collation where “a” != “?”
• MYSQL case sensitive search for utf8_bin field