The tag has no wiki summary.

learn more… | top users | synonyms

-2
votes
0answers
23 views

\06 is converted to a character that looks like a spade [migrated]

saveInPath = "C:\Users\User\Documents\AV_FILES\06_School\Python205\ProjectNov2014\CODE_CLEAN\NotQuiteBackUP" lccSysProcCons = SystemProcessControls() dateTimeNow = ...
1
vote
0answers
123 views

Implement a new Unicode encoding in a suitable open-source text-editor? [closed]

The very fact that the Unicode standard allows multiple encodings means that there's nothing inherently special about UTF-8/16/32. I want to use the Unicode character set, but I want to encode the ...
0
votes
3answers
142 views

Why does utf-8 waste several bits in it's encoding

According to the Wikipedia article, UTF-8 has this format: First code Last code Bytes Byte 1 Byte 2 Byte 3 Byte 4 point point Used U+0000 U+007F 1 0xxxxxxx U+0080 ...
4
votes
2answers
146 views

Is a newline convention not part of an encoding?

There are different conventions of representing the new line character in different types of OSes. Does the newline convention have nothing to do with what encoding is used? Is the newline ...
1
vote
1answer
60 views

String encoding in string instances from different languages

I recently got a requirement to develop a chat-like application, or rather, a foundation of classes and methods that would allow certain applications to have chat-like features. The framework must be ...
1
vote
1answer
272 views

How many bytes can I fit into a QR code, with low error correction?

I need to fit at least 300 bytes (or much much more) into a QR code and think I can do this by mapping each byte into the associated ISO/IEC 8859-1 character located here. Since each byte (1-255) ...
37
votes
2answers
5k views

Why do so many hashed and encrypted strings end in an equals sign?

I work in C# and MSSQL and as you'd expect I store my passwords salted and hashed. When I look at the hash stored in an nvarchar column (for example the out the box aspnet membership provider). I've ...
2
votes
3answers
306 views

When should I *not* use Unicode? [duplicate]

Unicode seems that its becoming more and more ubiquitous these days if it's not already, but I have to wonder if there are any domains were Unicode isn't the best implementation choice. Are there any ...
0
votes
2answers
337 views

When is it beneficial to not use utf-8? [duplicate]

When is it beneficial to use encodings other than UTF-8? Aside from dealing with pre-unicode documents, that is. And more importantly, why isn't UTF-8 the default in most languages? That is, why do I ...
2
votes
3answers
650 views

Should I convert the whole project to UTF-8?

I am working on a highly customized shop software, based on a open-source one, written in PHP and usual web techniques (CSS, HTML, JS). I did a lot of customization in the past months/years and ...
1
vote
1answer
162 views

How to cross-reference many character encodings with ASCII OR UTFx?

I'm working with a binary structure, the goal of which is to index the significance of specific bits for any character encoding so that we may trigger events while doing specific checks against the ...
2
votes
4answers
4k views

What encoding is used by javax.xml.transform.Transformer? [closed]

Please can you answer a couple of questions based on the code below (excludes the try/catch blocks), which transforms input XML and XSL files into an output XSL-FO file: File xslFile = new ...
3
votes
1answer
194 views

Encoding to ASCII where original encoding of string is not known

Given that we do not know the encoding of a string what is the best way to make sure that it is transformed to say ASCII? Also in such situations we are willing to accept potential loss of data.
1
vote
3answers
1k views

Is path in Set-Cookie URL encoded?

I'm writing some code that sets cookies and I'm wondering about the exact semantics of the Set-Cookie header. Imagine the following HTTP header line: Set-Cookie: name=value; Path=/%20 For with path ...
14
votes
2answers
42k views

How to detect the encoding of a file?

On my filesystem (Windows 7) I have some text files (These are SQL script files, if that matters). When opened with Notepad++, in the "Encoding" menu some of them are reported to have an encoding of ...
26
votes
7answers
2k views

Is the carriage-return char considered obsolete

I wrote an open source library that parses structured data but intentionally left out carriage-return detection because I don't see the point. It adds additional complexity and overhead for little/no ...
1
vote
2answers
2k views

What does the python Codecs module do?

I just read through the documentation on the Codecs module, but I guess my knowledge/experience of comp sci doesn't run deep enough yet for me to comprehend it. It's for dealing with ...
5
votes
2answers
725 views

How relevant is UTF-7 when it comes to parsing emails?

I recently implemented incoming emails for an application and boy, did I open the gates of hell? Since then every other day an email arrives that makes the app fail in a different way. One of those ...
4
votes
3answers
2k views

How does it matter if a character is 8 bit or 16 bit or 32 bit

Well, I am reading Programing Windows with MFC, and I came across Unicode and ASCII code characters. I understood the point of using Unicode over ASCII, but what I do not get is how and why is it ...
13
votes
3answers
29k views

Why do we need to put N before strings in Microsoft SQL Server?

I'm learning T-SQL. From the examples I've seen, to insert text in a varchar() cell, I can write just the string to insert, but for nvarchar() cells, every example prefix the strings with the letter ...
2
votes
2answers
225 views

What are the commonly confused encodings that may result in identical test data?

I'm fixing code that is using ASCIIEncoding in some places and UTF-8 encoding in other functions. Since we aren't using the UTF-8 features, all of our unit tests passed, but I want to create a ...
5
votes
3answers
860 views

Should my source code be in UTF-8?

I feel that often you don't really choose what format your code is in. I mean most of my tools in the past have decided for me. Or I haven't really even thought about it. I was using TextPad on ...
2
votes
2answers
298 views

What is really happening when we change encoding in a string?

http://php.net/manual/en/function.mb-convert-encoding.php Say I do: $encoded = mb_convert_encoding ($original); That looks like simple enough. WHat I am imagining is the following $original has a ...
4
votes
2answers
954 views

Why was ASCII needed?

With an encoding such as EBCDIC being in existence already (and being 8 bit to boot), what was the need to invent yet another encoding and a 7 bit one at that? Why was ASCII invented and what ...
3
votes
2answers
206 views

What steps can I take to avoid character encoding issues in a web application?

In previous web applications I've built, I've had issues with users entering exotic characters into forms which get stored strangely in the database, and sometimes appear different or double-encoded ...
3
votes
3answers
482 views

Was API hooking done as needed for Stuxnet to work? I don't think so

Caveat: I am a political science student and I have tried my level best to understand the technicalities; if I still sound naive please overlook that. In the Symantec report on Stuxnet, the authors ...
3
votes
3answers
250 views

what limitation will we face if each user-perceived character is assigned to one codepoint?

Hi all I was wondering what limitations will we have if Unicode had decided to assign one and only one codepoint to every user-perceived character? Currently, Unicode has code-points that correspond ...
2
votes
2answers
408 views

Should I HTML encode all output from my API?

I am creating a RESTful JSON API to access data from our website where the content is in German. A handful of the fields will return formatted HTML while most are single lines of text although they ...
1
vote
1answer
5k views

What is the Best Collation for Use in MySQL Tables?

I'm curious what is considered the standard today for use as the Collation of MySQL Tables? I was told that Latin-1 was the best choice when I was beginning with MySQL, but came across this post from ...
25
votes
5answers
32k views

What is the advantage of choosing ASCII encoding over UTF-8?

All characters in ASCII can be encoded using UTF-8 without an increase in storage (both requires a byte of storage). UTF-8 has the added benefit of character support beyond "ASCII-characters". If ...
9
votes
3answers
2k views

Is UTF-16 fixed-width or variable-width? Why doesn't UTF-8 have byte-order problem?

Is UTF-16 fixed-width or variable-width? I got different results from different sources: From http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF: UTF-16 stores Unicode characters in ...
1
vote
3answers
190 views

Does printing out numbers involve converting the numbers into characters?

In programming languages, when numbers (either integer, or real) are printed out, are they firstly converted to the codes of the readable characters that are meant to represent the numbers, and then ...
14
votes
4answers
1k views

What issues lead people to use Japanese-specific encodings rather than Unicode?

At work I come across a lot of Japanese text files in Shift-JIS and other encodings. It causes many mojibake (unreadable character) problems for all computer users. Unicode was intended to solve this ...
1
vote
3answers
153 views

Is it ok to use localized character encodings for code?

Is it ok to use a localized encoding (i.e. ISO-8859-15) instead of ANSI/UTF-8/some standard for code? What when you have names that do not translate very well into english code? Or when your company ...
0
votes
2answers
535 views

Encoding issues from MySQL database to PHP page

So I've been working on a small, small side project to learn a bit more about PHP database interactions. I bought a small, cheap database of "Inspirational Stories" and set to work on turning it into ...
22
votes
7answers
2k views

Should character encodings besides UTF-8 (and maybe UTF-16/UTF-32) be deprecated?

A pet peeve of mine is looking at so many software projects that have mountains of code for character set support. Don't get me wrong, I'm all for compatibility, and I'm happy that text editors let ...