Unicode is intended to be a universal character set for describing all the characters required for written text incorporating all writing systems, technical symbols and punctuation.
12
votes
3answers
277 views
A Unicode sentinel value I can use?
I am desiging a file format and I want to do it right. Since it is a binary format, the very first byte (or bytes) of the file should not form valid textual characters (just like in the PNG file ...
2
votes
2answers
100 views
Strategy for website with international strings
What things need to be considered for a Website that contains International strings, for instance Simplified Chinese and English mixed.
UTF8 seems to me a natural choice, including a meta tag. ...
4
votes
1answer
239 views
Is O(1) random access into variable length encoding strings useful?
I remember reading that there are no existing data structures which allow for random-access into a variable length encoding, like UTF-8, without requiring additional lookup tables.
The main question ...
13
votes
2answers
1k views
Why does Java use UTF-16 for internal string representation?
I would imagine the reason was fast, array like access to the character at index, but some characters won't fit into 16 bits, so it wouldn't work...
So if you have to handle special cases anyways, ...
32
votes
2answers
785 views
Unicode license
The Unicode Terms of use state that any software that uses their data files (or a modification of it) should carry the Unicode license references.
It seems to me that most Unicode libraries have ...
3
votes
1answer
431 views
understanding the encoding scheme in python 3
I got this error in my program which grab data from different website and write them to a file:
'charmap' codec can't encode characters in position 151618-151624: character maps to <undefined>
...
2
votes
3answers
581 views
How does it matter if a character is 8 bit or 16 bit or 32 bit
Well, I am reading Programing Windows with MFC, and I came across Unicode and ASCII code characters. I understood the point of using Unicode over ASCII, but what I do not get is how and why is it ...
9
votes
3answers
5k views
Why do we need to put N before strings in Microsoft SQL Server?
I'm learning T-SQL. From the examples I've seen, to insert text in a varchar() cell, I can write just the string to insert, but for nvarchar() cells, every example prefix the strings with the letter ...
6
votes
1answer
375 views
How in the earth CHRW produce unicode codes given that it only accept 65k possible input?
http://babelstone.blogspot.com/2005/11/how-many-unicode-characters-are-there.html says there are 1 million unicode characters and around 240k of which are already assigned.
1 million > 240k > 65k
...
4
votes
2answers
133 views
Prerequisites for developing an application with Unicode support [closed]
What could be the necessary prerequisites to be taken when developing an application with Unicode support in the context of
Web applications
Desktop applications
Embedded applications
...
3
votes
3answers
207 views
what limitation will we face if each user-perceived character is assigned to one codepoint?
Hi all I was wondering what limitations will we have if Unicode had decided to assign one and only one codepoint to every user-perceived character?
Currently, Unicode has code-points that correspond ...
8
votes
7answers
500 views
What's the point of adding Unicode identifier support to various language implementations?
I personally find reading code full of Unicode identifiers confusing. In my opinion, it also prevents the code from being easily maintained. Not to mention all the effort required for authors of ...
0
votes
2answers
176 views
Unicode clarification
Why is the length of the some characters e.g. the following 'ᨒ' 3 when it should be 2
ᨒ
U+1a12
1a12 means 6674
2^16 is 65536 so 6674 should take only 2 bytes and not three
1
vote
3answers
156 views
Unicode Explanation Required
Can someone explain what this means?
Unicode defines a codespace of 1,114,112 code points in the range 0hex to 10FFFFhex.
http://en.wikipedia.org/wiki/Unicode
7
votes
3answers
1k views
Is UTF-16 fixed-width or variable-width? Why doesn't UTF-8 have byte-order problem?
Is UTF-16 fixed-width or variable-width? I got different results
from different sources:
From http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF:
UTF-16 stores Unicode characters in ...
13
votes
4answers
645 views
What issues lead people to use Japanese-specific encodings rather than Unicode?
At work I come across a lot of Japanese text files in Shift-JIS and other encodings. It causes many mojibake (unreadable character) problems for all computer users. Unicode was intended to solve this ...
35
votes
9answers
1k views
Why are there multiple Unicode encodings?
I thought Unicode was designed to get around the whole issue of having lots of different encoding due to a small address space (8 bits) in most of the prior attempts (ASCII, etc.).
Why then are there ...
4
votes
3answers
376 views
What should I consider when converting from ASCII to Unicode?
For instance if I had only ASCII characters and then switched to Unicode now I have to consider special characters, and bigger strings or chars. What else along these lines should I be considering?
...
2
votes
3answers
2k views
What does the lack of Unicode support in PHP mean?
How can the lack of Unicode support in PHP affect a PHP web app?
3
votes
1answer
482 views
18
votes
7answers
1k views
Should character encodings besides UTF-8 (and maybe UTF-16/UTF-32) be deprecated?
A pet peeve of mine is looking at so many software projects that have mountains of code for character set support. Don't get me wrong, I'm all for compatibility, and I'm happy that text editors let ...
3
votes
2answers
118 views
More knowledge about Unicode and Character sets
Just finished reading Joels excellent article on Unicode and Character sets, buts its left me with a feeling that I don't fully understand the subject. Can you point me to other good articles that ...
12
votes
4answers
791 views
Why exactly can't PHP have full unicode support?
Everybody knows, that PHP has problems with Unicode. Version 6 is effectively abandoned, because of Unicode implementation difficulties. But I wonder if anyone knows what are the exact reasons? ...
30
votes
14answers
3k views
Is it bad to use Unicode characters in variable names?
I recently tried to implement a ranking algorithm, AllegSkill, to Python 3.
Here's what the maths looks like:
No, really.
This is then what I wrote:
t = (µw-µl)/c # those are used in
e ...
329
votes
22answers
37k views
Should UTF-16 be considered harmful?
I'm going to ask what is probably quite a controversial question: "Should one of the most
popular encodings, UTF-16, be considered harmful?"
Why do I ask this question?
How many programmers are ...
