How to detect wrongly encoded strings in PHP

Question

Suppose to have string like this one:

Çë½ÌÎªÊ²Ã´ÎÒÒ»½øÐÐ²é¶¾Ê±¾Í·¢ÏÖÏµÍ³ÅÌ¿ÉÓÃ¿Õ ¼ä¾Í¼±¾ç¼õÉÙ£

They have been wrongly encoded. How do you think it's possible to know if in fact, it's wrongly encoded? An example of right encoded string would be

Ciao mamm@ guardà come mi divertò

I tough that there are 2 mayor differences among the 2:

Numer of whitespaces/string_lenght
Number of vocals(aeiou)/string_lenght

Than the code would be something like

if({Numer of whitespaces/string_lenght} < 0.05 (1 every 20 other characters)}
     OR {Number of vocals(aeiou)/string_lenght} < 0.2 (1 every 5 other characters)})
  return WRONG
else
  return OK

Do you have any better idea? Maybe there is some php function already tested that fits my case? Thanks!

What, exactly, counts as "wrongly encoded"? Do you mean that character data is reported (e.g. in an HTTP header, or the character set of a database column) to have one encoding, but should have another? Is the sample string supposed to be the latin1 code points of the characters shown, or the UTF-8 code points that actually appear in the page? — outis
– outis, Commented Dec 29, 2011 at 16:23

outis · Accepted Answer · 2011-12-29 14:09:20Z

1

If you know what the encoding should be, use mb_check_encoding. If you don't know what the encoding should be, try mb_detect_encoding, which returns FALSE if no valid encoding is found.

answered Dec 29, 2011 at 14:09

outis

77.7k23 gold badges155 silver badges228 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

cedivad Over a year ago

No, that won't work. The string in this case are valid latin1, but they are meaningless. Also mb_check_encoding don't check if every character is valid, it only checks if the stream is valid ;)

symcbean Over a year ago

Then you'd need to base it on the frequency of characters in the expected language and or use pspell

Collectives™ on Stack Overflow

How to detect wrongly encoded strings in PHP

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related