I'm kind of horrified that people actually suggest using UTF-8 inside a std::string
and are OK with UTF-16 std::wstring
.
Do you actually do this? Do you think that it is OK?
I'm kind of horrified that people actually suggest using UTF-8 inside a std::string
and are OK with UTF-16 std::wstring
.
Do you actually do this? Do you think that it is OK?
My convention is:
std::string
= UTF-8. For general use.Utf16String
= UTF-16. Used mainly for Windows system calls.
std::wstring
in Windowsstd::basic_string<uint16_t>
in LinuxUtf32String
= UTF-32.
std::basic_string<uint32_t>
in Windowsstd::wstring
in Linux.std::vector<unsigned char>
= non-string binary dataYou just have to get used to thinking of strings as being an array of code points instead of an array of characters. With this view, all string methods work correctly except for find_first_of
/find_last_of
/find_first_not_of
/find_last_not_of
.
Well, let's say that many programmers still don't know that UTF-16 is a multibyte encoding (they still think that 16-bit wide char is enough to represent all the Unicode characters, but actually they are stuck with the old UCS-2).
However, there's no real drawback in using a wstring
for storing UCS-16 text, but you should know that its length
doesn't necessarily represent the number of text symbols that will be printed.
vector
instead of base_string
? Having a string
implies, that methods actually work.
Oct 29, 2010 at 17:27
Even if you store UCS-4 in a string, you still can't guarantee a 1:1 relationship between an item stored in the string and (at least a typical user will view as) a single character when it's displayed or printed (if the string happens to contain any combining character sequences).
Now, it's true that with UCS-4 a single item in the string maps to a single code-point even if that's not necessarily a single character.You can avoid multiple string elements mapping to a single code point. You can't (entirely) avoid multiple code points mapping to a single "character" (and canonicalization to even get close is non-trivial). The problem is that "code point" is an almost entirely artificial construct. Getting excited about/condemning one while ignoring the other doesn't (IMO) make a whole lot of sense.