The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
4 views

Why do some Unicode characters appear as boxes? [migrated]

Some Unicode characters cannot be shown in my browser. I generated valid Unicode (utf8) characters but many of them are unable to be displayed. If they don't have representation, then why are they ...
35
votes
2answers
4k views

Why do so many hashed and encrypted strings end in an equals sign?

I work in C# and MSSQL and as you'd expect I store my passwords salted and hashed. When I look at the hash stored in an nvarchar column (for example the out the box aspnet membership provider). I've ...
1
vote
3answers
235 views

When should I *not* use Unicode? [duplicate]

Unicode seems that its becoming more and more ubiquitous these days if it's not already, but I have to wonder if there are any domains were Unicode isn't the best implementation choice. Are there any ...
0
votes
2answers
319 views

When is it beneficial to not use utf-8? [duplicate]

When is it beneficial to use encodings other than UTF-8? Aside from dealing with pre-unicode documents, that is. And more importantly, why isn't UTF-8 the default in most languages? That is, why do I ...
2
votes
3answers
492 views

Should I convert the whole project to UTF-8?

I am working on a highly customized shop software, based on a open-source one, written in PHP and usual web techniques (CSS, HTML, JS). I did a lot of customization in the past months/years and ...
1
vote
1answer
157 views

How to cross-reference many character encodings with ASCII OR UTFx?

I'm working with a binary structure, the goal of which is to index the significance of specific bits for any character encoding so that we may trigger events while doing specific checks against the ...
0
votes
0answers
114 views

Robustness Principle, Building Solid API's, and URLEncoding

One thing that has continually plagued my fellow colleagues and I are "special characters". These are things like ", #, &, \, @, and any other character that one might include in a string literal ...
1
vote
4answers
3k views

What encoding is used by javax.xml.transform.Transformer? [closed]

Please can you answer a couple of questions based on the code below (excludes the try/catch blocks), which transforms input XML and XSL files into an output XSL-FO file: File xslFile = new ...
3
votes
1answer
180 views

Encoding to ASCII where original encoding of string is not known

Given that we do not know the encoding of a string what is the best way to make sure that it is transformed to say ASCII? Also in such situations we are willing to accept potential loss of data.
1
vote
3answers
830 views

Is path in Set-Cookie URL encoded?

I'm writing some code that sets cookies and I'm wondering about the exact semantics of the Set-Cookie header. Imagine the following HTTP header line: Set-Cookie: name=value; Path=/%20 For with path ...
8
votes
2answers
21k views

How to detect the encoding of a file?

On my filesystem (Windows 7) I have some text files (These are SQL script files, if that matters). When opened with Notepad++, in the "Encoding" menu some of them are reported to have an encoding of ...
26
votes
7answers
2k views

Is the carriage-return char considered obsolete

I wrote an open source library that parses structured data but intentionally left out carriage-return detection because I don't see the point. It adds additional complexity and overhead for little/no ...
1
vote
2answers
2k views

What does the python Codecs module do?

I just read through the documentation on the Codecs module, but I guess my knowledge/experience of comp sci doesn't run deep enough yet for me to comprehend it. It's for dealing with ...
5
votes
2answers
610 views

How relevant is UTF-7 when it comes to parsing emails?

I recently implemented incoming emails for an application and boy, did I open the gates of hell? Since then every other day an email arrives that makes the app fail in a different way. One of those ...
4
votes
3answers
2k views

How does it matter if a character is 8 bit or 16 bit or 32 bit

Well, I am reading Programing Windows with MFC, and I came across Unicode and ASCII code characters. I understood the point of using Unicode over ASCII, but what I do not get is how and why is it ...
13
votes
3answers
21k views

Why do we need to put N before strings in Microsoft SQL Server?

I'm learning T-SQL. From the examples I've seen, to insert text in a varchar() cell, I can write just the string to insert, but for nvarchar() cells, every example prefix the strings with the letter ...
2
votes
2answers
215 views

What are the commonly confused encodings that may result in identical test data?

I'm fixing code that is using ASCIIEncoding in some places and UTF-8 encoding in other functions. Since we aren't using the UTF-8 features, all of our unit tests passed, but I want to create a ...
5
votes
3answers
771 views

Should my source code be in UTF-8?

I feel that often you don't really choose what format your code is in. I mean most of my tools in the past have decided for me. Or I haven't really even thought about it. I was using TextPad on ...
2
votes
2answers
275 views

What is really happening when we change encoding in a string?

http://php.net/manual/en/function.mb-convert-encoding.php Say I do: $encoded = mb_convert_encoding ($original); That looks like simple enough. WHat I am imagining is the following $original has a ...
4
votes
2answers
735 views

Why was ASCII needed?

With an encoding such as EBCDIC being in existence already (and being 8 bit to boot), what was the need to invent yet another encoding and a 7 bit one at that? Why was ASCII invented and what ...
3
votes
2answers
179 views

What steps can I take to avoid character encoding issues in a web application?

In previous web applications I've built, I've had issues with users entering exotic characters into forms which get stored strangely in the database, and sometimes appear different or double-encoded ...
3
votes
3answers
464 views

Was API hooking done as needed for Stuxnet to work? I don't think so

Caveat: I am a political science student and I have tried my level best to understand the technicalities; if I still sound naive please overlook that. In the Symantec report on Stuxnet, the authors ...
3
votes
3answers
234 views

what limitation will we face if each user-perceived character is assigned to one codepoint?

Hi all I was wondering what limitations will we have if Unicode had decided to assign one and only one codepoint to every user-perceived character? Currently, Unicode has code-points that correspond ...
2
votes
2answers
349 views

Should I HTML encode all output from my API?

I am creating a RESTful JSON API to access data from our website where the content is in German. A handful of the fields will return formatted HTML while most are single lines of text although they ...
1
vote
1answer
4k views

What is the Best Collation for Use in MySQL Tables?

I'm curious what is considered the standard today for use as the Collation of MySQL Tables? I was told that Latin-1 was the best choice when I was beginning with MySQL, but came across this post from ...
20
votes
5answers
24k views

What is the advantage of choosing ASCII encoding over UTF-8?

All characters in ASCII can be encoded using UTF-8 without an increase in storage (both requires a byte of storage). UTF-8 has the added benefit of character support beyond "ASCII-characters". If ...
8
votes
3answers
2k views

Is UTF-16 fixed-width or variable-width? Why doesn't UTF-8 have byte-order problem?

Is UTF-16 fixed-width or variable-width? I got different results from different sources: From http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF: UTF-16 stores Unicode characters in ...
1
vote
3answers
190 views

Does printing out numbers involve converting the numbers into characters?

In programming languages, when numbers (either integer, or real) are printed out, are they firstly converted to the codes of the readable characters that are meant to represent the numbers, and then ...
13
votes
4answers
994 views

What issues lead people to use Japanese-specific encodings rather than Unicode?

At work I come across a lot of Japanese text files in Shift-JIS and other encodings. It causes many mojibake (unreadable character) problems for all computer users. Unicode was intended to solve this ...
1
vote
3answers
148 views

Is it ok to use localized character encodings for code?

Is it ok to use a localized encoding (i.e. ISO-8859-15) instead of ANSI/UTF-8/some standard for code? What when you have names that do not translate very well into english code? Or when your company ...
0
votes
2answers
505 views

Encoding issues from MySQL database to PHP page

So I've been working on a small, small side project to learn a bit more about PHP database interactions. I bought a small, cheap database of "Inspirational Stories" and set to work on turning it into ...
19
votes
7answers
2k views

Should character encodings besides UTF-8 (and maybe UTF-16/UTF-32) be deprecated?

A pet peeve of mine is looking at so many software projects that have mountains of code for character set support. Don't get me wrong, I'm all for compatibility, and I'm happy that text editors let ...