The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
3answers
131 views

Why does utf-8 waste several bits in it's encoding

According to the Wikipedia article, UTF-8 has this format: First code Last code Bytes Byte 1 Byte 2 Byte 3 Byte 4 point point Used U+0000 U+007F 1 0xxxxxxx U+0080 ...
0
votes
2answers
334 views

When is it beneficial to not use utf-8? [duplicate]

When is it beneficial to use encodings other than UTF-8? Aside from dealing with pre-unicode documents, that is. And more importantly, why isn't UTF-8 the default in most languages? That is, why do I ...
1
vote
1answer
478 views

UTF-16 Pitfalls, Chinese

I'm going to be writing an application that is pure HTML5 and JS and MVC.net back-end. We have .resx files that are getting compiled to .js files for resources in the html5 application. The ...
12
votes
2answers
38k views

How to detect the encoding of a file?

On my filesystem (Windows 7) I have some text files (These are SQL script files, if that matters). When opened with Notepad++, in the "Encoding" menu some of them are reported to have an encoding of ...
430
votes
2answers
56k views

Is the use of “utf8=✓” preferable to “utf8=true”?

I have recently seen a few URIs containing the query parameter "utf8=✓". My first impression (after thinking "mmm, looks cool") was that this could be used to detect a broken character encoding. So, ...
2
votes
2answers
221 views

What are the commonly confused encodings that may result in identical test data?

I'm fixing code that is using ASCIIEncoding in some places and UTF-8 encoding in other functions. Since we aren't using the UTF-8 features, all of our unit tests passed, but I want to create a ...
5
votes
3answers
838 views

Should my source code be in UTF-8?

I feel that often you don't really choose what format your code is in. I mean most of my tools in the past have decided for me. Or I haven't really even thought about it. I was using TextPad on ...
3
votes
2answers
202 views

What steps can I take to avoid character encoding issues in a web application?

In previous web applications I've built, I've had issues with users entering exotic characters into forms which get stored strangely in the database, and sometimes appear different or double-encoded ...
25
votes
5answers
30k views

What is the advantage of choosing ASCII encoding over UTF-8?

All characters in ASCII can be encoded using UTF-8 without an increase in storage (both requires a byte of storage). UTF-8 has the added benefit of character support beyond "ASCII-characters". If ...
9
votes
3answers
2k views

Is UTF-16 fixed-width or variable-width? Why doesn't UTF-8 have byte-order problem?

Is UTF-16 fixed-width or variable-width? I got different results from different sources: From http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF: UTF-16 stores Unicode characters in ...
21
votes
7answers
2k views

Should character encodings besides UTF-8 (and maybe UTF-16/UTF-32) be deprecated?

A pet peeve of mine is looking at so many software projects that have mountains of code for character set support. Don't get me wrong, I'm all for compatibility, and I'm happy that text editors let ...