How can the lack of Unicode support in PHP affect a PHP web app?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Any website that purports to be multi-lingual or to deal with documents or content that is not representable in Latin-1 is likely to be problematic if you don't have Unicode support.
Another problematic use-case is when content might contain mathematical and other symbols.
However, your example of Facebook suggests that in fact you can in fact "do" Unicode in PHP. Alternatively,
and has lots of UTF-8 content.
OK, here's what the PHP doc for "String" says:
So PHP does have Unicode support. It is just that "native strings" are not Unicode based.
So what it means is that if you need to deal with any language (or set of languages) that cannot be encode in an 8-bit character set, your PHP code is going to be more cumbersome at any point where it needs to process content as (real) characters.
There are (at least) three different approaches to text in the modern programming world.
In general a language and/or the libraries, APIs and protocols used with it encourage or even force a given approach.
php takes the first approach. A string is just a sequence of bytes, there is no special type for Unicode. Some would characterise this as "not supporting unicode" but I would consider that a mis-characterisation.
What is does mean is that if you are using php with utf-8 it is your responsibility to do so correctly. UTF-8 is a fairly well behaved encoding so you don't have to worry as much as you did with some legacy multibyte encodings but you still need to be careful about some operations. Especially truncation (don't want to cut in the middle of a multibyte sequence) and character counting (is a count of utf-8 code units acceptable for the given application or do you need to count code points or even something else). If you don't take responsibility for these things your users are likely to see glitches.
On the other hand it matches well with how internet protocols work. A system that works with strings as sequences of bytes can deal with internet data without encoding conversions. A system that works in seuences of code points or utf-16 code units must usually perform conversions of text going to/from the internet.
Actually php has methods to manipulate multi byte strings - please see mbstring. There is also a configuration option in php.ini to use mbstring replacements for most (all) of the string manipulation functions - for details on that please see mbstring overload.
It means that you have to take some shortcuts and do nasty tricks in order to get unicode. And that those tricks are going to make the code more cumbersome and less readable.