Reputation
360
Top tag
Next privilege 500 Rep.
Access review queues
Badges
3 12
Newest
 Pundit
Impact
~79k people reached

  • 0 posts edited
  • 3 helpful flags
  • 113 votes cast
Dec
31
comment Should UTF-16 be considered harmful?
BTW, I've just checked editing these letters, they don't give me a problems neither in Opera, nor in Windows 7. Opera seems to edit them properly, so does Notepad. File with these letters in the name has been created successfully.
Dec
29
comment Should UTF-16 be considered harmful?
Well, as I mentioned earlier, I didn't find this post very convincing either. This post goes into details of handling UTF-16 in certain APIs or languages. If the software doesn't handle the standard properly, that's a problem. But what's wrong with the encoding itself anyway? If some software implements only half of the standard, that's not the standard's problem.
Apr
2
comment Should UTF-16 be considered harmful?
Theoretically, yes. In practice there are such things as, say, UTF-16BE, which means UTF-16 in big endian without BOM. This is not some thing I made up, this is an actual encoding allowed in ID3v2.4 tags (ID3v2 tags suck, but are, unfortunately, widely used). And in such cases you have to define endianness externally, because the text itself doesn't contain BOM. UTF-8 is always written one way and it doesn't have such a problem.
Mar
7
comment Should UTF-16 be considered harmful?
I'm sorry, I didn't really get the idea why transition to UTF-8 should be less painful. I also think that inconsistency in C++ makes it worse. Say, Java is very specific on the characters: char[] is no more than a char array, String is a string and Character is a character. Meanwhile, C++ is a mess with all the new stuff added to an existing language. To my mind, they should've abandoned any backwards compablity and design C++ in the way that doesn't allow to mix up structural programming and OOP or Unicode and other encodings. Not that I want to start a holy war, that's merely my opinion.
Feb
20
comment Should UTF-16 be considered harmful?
Well, what I meant is that you don't have to worry about byte order. UTF-8 can have a BOM indeed (it is actually UTF-16 big endian BOM encoded in 3 bytes), though it's neither required, nor recommended according to the standard. As for the APIs I think the problem is that they were designed when surrogate pairs were either non-existent yet, or not really adopted. And when something gets patched up, it's always not as good as redesigning from the scratch. The only (painful) way is to drop any backwards compability and redesign the APIs. Should they switch to UTF-8 in the process, I don't know.
Feb
20
comment Should UTF-16 be considered harmful?
Well, if I had to choose between UTF-8 and UTF-16, I would definitely stick to UTF-8 as it has no BOM, ASCII-compliant and has the same encoding scheme for any plane. But I have to admit that UTF-16 is simpler and more efficient for most BMP characters. There's nothing worng with UTF-16 except the psychological aspects (mostly fixed-size isn't fixed size). Sure, one encoding would be better, but since both UTF-8 and UTF-16 are widely used, they have their advantages.
Sep
26
comment Should UTF-16 be considered harmful?
If BMP is that far from having enough capacity to write normally in Chinese, how do they manage to write in such encodings as GBK or GB 2312? It is clear that support of other planes would be useful, but nonetheless.
Aug
12
comment Should UTF-16 be considered harmful?
It does, but characters outside BMP are not for everyday use, they can be used, for example, for old texts or to write names with rare hieroglyphs in them. And all characters that are commonly used fit into BMP.
Jul
2
comment Should UTF-16 be considered harmful?
There is, but it's not really a problem specific for Unicode since standart encodings also don't include this characters. People use homophones and other ways to write such names, and that can be done in any encoding, including Unicode. Probably there are serious difficulties even with inputting rare symbols, so the situation doesn't happen all of a sudden and users won't be surprised to find out the program is refusing to accept them correctly if it does.
Jun
29
comment Should UTF-16 be considered harmful?
And what did I write wrong? All characters of plane 2 contain only rare or historic symbols and all other characters fit into BMP and thus don't need surrogate pairs.
Jun
27
comment Should UTF-16 be considered harmful?
Not true, Asian scripts like Japanese, Chinese or Arabic belong to BMP also. BMP itself is actually very large and certainly large enough to include all the scripts used nowadays, it's not like it includes only European scripts or something. No, if you are really going to encounter non-BMP characters, you'll almost definitely know it.
Jun
26
comment Should UTF-16 be considered harmful?
UTF-16 is simpler for anything inside BMP, that's why it is used so widely. But I'm a fan of UTF-8 too, it also has no problems with byte order, which works to its advantage.
Jun
26
comment Should UTF-16 be considered harmful?
I agree with the last edit. The simplest example: we still use C and C++ though both languages use pointers and thus are not safe.
Jun
26
comment Should UTF-16 be considered harmful?
Actually I'm from Russia and encounter cyrillics all the time (including my own programs), so I don't think that I have Anglo-centric view. :) Mentioning ASCII is not quite appropirate, because it's not Unicode and doesn't support specific characters. UTF-8, UTF-16, UTF-32 support the very same international character sets, they are just intended for use in their specific areas. And this is exactly my point: if you use mostly English, use UTF-8, if you use mostly cyrillics, use UTF-16, if you use ancient languages, use UTF-32. Quite simple.
Jun
26
comment Should UTF-16 be considered harmful?
Certainly. But that doesn't mean that if someone can use something incorrectly, we shouldn't use it at all, right?