341 reputation
212
bio website malcolm-soft.com
location New York, United States
age 25
visits member for 3 years, 4 months
seen yesterday

Mobile application developer who freaking loves what he does - that's the most apt description for me. I mainly spend time writing something for Android, but I also play a lot with other technologies as well.


Aug
12
comment Should UTF-16 be considered harmful?
Consider it non-Unicode, that would clear the confusion. Historically this method is simply not meant to work reliably in conditions where I18N is required, it is not even locale-aware. Java has different facilities to do that. And I have to remind you that we're off the topic, still talking about subtleties of APIs, not about the UTF-16 itself.
Aug
12
comment Should UTF-16 be considered harmful?
Yes, equalsIgnoreCase() compares chars, not codepoints, and it is stated in the docs. I certainly agree that if this method compared strings usign code points, it would be much simpler. But this isn't a problem of UTF-16 itself, it is a problem of a platform which was originally designed for UCS-2 - exactly what I'm talking about.
Aug
12
comment Should UTF-16 be considered harmful?
@tchrist This is all too general. First of all, define what is a "BMP screwup" and where you look for them. Also, I've already said that there's a huge difference between something that was designed for UTF-16 and something that was designed for UCS-2 and then switched to UTF-16. And if we put aside API and language deficiencies, I don't reallly see what's so terribly difficult in handling surrogate pairs, especially in comparison with UTF-8.
Aug
11
comment Should UTF-16 be considered harmful?
@tchrist Actually I think that most problems appear from the dated software which was designed for UCS-2. If you implement the standard today, you will be almost certainly aware that UTF-16 has a concept of surrogate pairs, since it is written about almost everywhere, starting with Wikipedia. And if you are an ignorant developer, you may not implement support for characters outside BMP in UTF-8 as well. Or even treat every text as if each byte represented only one character.
Aug
1
comment If a variable has getter and setter, should it be public?
+1 for "you should be asking an object to do an operation for you"
Aug
1
awarded  Supporter
Jul
28
comment Are languages just syntax or do they include the framework too?
No, no, it doesn't dictate implementation details and there are not many classes which are explicitly mentioned in the specs compared to the core library in its entirety. But my point is that the language and the library are tightly connected. A vendor actually can't not ship at least the most necessary parts of the library because the spec makes their presence mandatory. And therefore these parts become sort of a part of the language.
Jul
28
comment Are languages just syntax or do they include the framework too?
I mean the latter, of course. Say, "The direct superclass of an enum type named E is Enum<E>." Enum is a class from the core library. Or: "The unchecked exceptions classes are the class RuntimeException and its subclasses, and the class Error and its subclasses. All other exception classes are checked exception classes." And this directly influences how particular exceptions must be handled in the terms of language. And such mentions happen throughout the whole specification.
Jul
28
comment Are languages just syntax or do they include the framework too?
What if the language specification frequently references the framework? This actually happens in Java Language Specification, for example.
Jul
28
awarded  Autobiographer
Feb
13
comment Should UTF-16 be considered harmful?
Why would they be angry?
Jan
23
comment Should UTF-16 be considered harmful?
Exactly, all the encodings cover all the code points; and as for the lack of available codes, I don't see how this can be possible in forseeable future. Most supplementary planes are still unused, and even the used ones aren't full yet. So given the total sizes of the known writing systems left, it is very possible that most planes will never be used, unless they start to use code points for something different than writing systems. By the way, UTF-8 can theoretically include 6-byte sequences, so it can represent even more code points than UTF-32, but what's the point?
Jan
1
comment Should UTF-16 be considered harmful?
Well, I don't think that mere existence of crappy implementations indicates harmfulness of the standard at all. :p This is just an update on the current situation: how problematic characters beyond BMP in Windows (and Opera) are now.
Jan
1
comment Should UTF-16 be considered harmful?
Once. I specially checked for this issue, and in Windows 7 the problem with the characters beyond BMP seems to be gone. Maybe this problem had been solved even in Vista.
Dec
31
comment Should UTF-16 be considered harmful?
BTW, I've just checked editing these letters, they don't give me a problems neither in Opera, nor in Windows 7. Opera seems to edit them properly, so does Notepad. File with these letters in the name has been created successfully.
Dec
29
comment Should UTF-16 be considered harmful?
Well, as I mentioned earlier, I didn't find this post very convincing either. This post goes into details of handling UTF-16 in certain APIs or languages. If the software doesn't handle the standard properly, that's a problem. But what's wrong with the encoding itself anyway? If some software implements only half of the standard, that's not the standard's problem.
Apr
2
comment Should UTF-16 be considered harmful?
Theoretically, yes. In practice there are such things as, say, UTF-16BE, which means UTF-16 in big endian without BOM. This is not some thing I made up, this is an actual encoding allowed in ID3v2.4 tags (ID3v2 tags suck, but are, unfortunately, widely used). And in such cases you have to define endianness externally, because the text itself doesn't contain BOM. UTF-8 is always written one way and it doesn't have such a problem.
Mar
7
comment Should UTF-16 be considered harmful?
I'm sorry, I didn't really get the idea why transition to UTF-8 should be less painful. I also think that inconsistency in C++ makes it worse. Say, Java is very specific on the characters: char[] is no more than a char array, String is a string and Character is a character. Meanwhile, C++ is a mess with all the new stuff added to an existing language. To my mind, they should've abandoned any backwards compablity and design C++ in the way that doesn't allow to mix up structural programming and OOP or Unicode and other encodings. Not that I want to start a holy war, that's merely my opinion.
Feb
20
comment Should UTF-16 be considered harmful?
Well, what I meant is that you don't have to worry about byte order. UTF-8 can have a BOM indeed (it is actually UTF-16 big endian BOM encoded in 3 bytes), though it's neither required, nor recommended according to the standard. As for the APIs I think the problem is that they were designed when surrogate pairs were either non-existent yet, or not really adopted. And when something gets patched up, it's always not as good as redesigning from the scratch. The only (painful) way is to drop any backwards compability and redesign the APIs. Should they switch to UTF-8 in the process, I don't know.
Feb
20
comment Should UTF-16 be considered harmful?
Well, if I had to choose between UTF-8 and UTF-16, I would definitely stick to UTF-8 as it has no BOM, ASCII-compliant and has the same encoding scheme for any plane. But I have to admit that UTF-16 is simpler and more efficient for most BMP characters. There's nothing worng with UTF-16 except the psychological aspects (mostly fixed-size isn't fixed size). Sure, one encoding would be better, but since both UTF-8 and UTF-16 are widely used, they have their advantages.