Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 378 down vote favorite
167

I have recently seen a few URIs containing the query parameter "utf8=✓". My first impression (after thinking "mmm, looks cool") was that this could be used to detect a broken character encoding.

So, is this a better way to resolve potential problems with character encoding, or is it just a developer having fun with a hack?

share|improve this question
11  
Fun fact: If it has query parameters, it's a URL. The only point of calling something a "URI" is if URNs are allowable values (spoiler alert: they never, ever are). – Mu Mind Oct 19 '12 at 3:35
5  
I disagree. There are schemes out there that look like URNs and that take query parameters - such as Bitcoin. URIs are not confined to browsers. See en.wikipedia.org/wiki/URI_scheme. This question may also address the general case where character encoding is required when a browser accesses a protocol handler. – Gary Rowe Oct 19 '12 at 8:29
1  
Give examples of these URLs or didn't happen. – hakre Oct 22 '12 at 12:59
5  
Off topic, but OK. Here's my personal donation Bitcoin URI: bitcoin:1KzTSfqjF2iKCduwz59nv2uqh1W2JsTxZH?amount=0.5&label=Agile%20Stack. Notice that the scheme is essentially a URN with query parameters, but it hands off to a protocol handler. This kind of URI could probably benefit from the “utf8=✓” workaround as well. – Gary Rowe Oct 22 '12 at 17:47

3 Answers

up vote 538 down vote accepted

By default, older versions of IE (<=8) will submit form data in Latin-1 encoding if possible. By including a character that can't be expressed in Latin-1, IE is forced to use UTF-8 encoding for its form submissions, which simplifies various backend processes, for example database persistence.

If the parameter was instead utf8=true then this wouldn't trigger the UTF-8 encoding in these browsers.

share|improve this answer
43  
I didn't know that, really interesting! – Florian Margaine Oct 13 '12 at 12:50
6  
Ah, so it's an IE workaround... nice. Saves a lot of validation too. – Gary Rowe Oct 13 '12 at 13:03
7  
@LarsViklund I should have been clearer with my comment. I meant that the validation associated with character encoding is simplified, not bypassed. – Gary Rowe Oct 13 '12 at 13:48
3  
@Lars Correct, it doesn't absolve you from having to check your input. But it does mean that encoding tweaks only become part of your security handling and don't taint the concept of your "standard processing" path – Gareth Oct 14 '12 at 10:08
20  
Also see stackoverflow.com/questions/3222013/…. Apparently Ruby on Rails used to use a snowman character, and was changed to a checkmark which was less ambiguous but less funny. – Jack V. Oct 17 '12 at 10:06
show 11 more comments
up vote 1 down vote

Another approach: specify the encoding in the head: . And use JS encodeURI, php urlencode before sending the get/post data.

share|improve this answer
8  
And when JavaScript is disabled what do you do then? – Gary Rowe Oct 18 '12 at 20:01
11  
You tell the client to upgrade their browser... – quickshiftin Oct 19 '12 at 19:25
2  
Please think of the poor NoScript users. – jah Nov 20 '12 at 19:08
up vote -3 down vote

You should only use the first half of the ascii characters to define encoding since these don't change based on used encoding. Using utf8 encoded characters before even knowing what encoding to use is IMHO flat out bad design.

share|improve this answer
1  
The ASCII character set only defines 128 characters, and all of them are encoded the same way in UTF-8 and ASCII. There are a great many extended character sets each defining 256 characters, of which the first 128 (which is I think what you're referring to) correspond exactly to ASCII. (I apologise for my previous comment, which was unhelpful) – John Bartholomew Oct 21 '12 at 18:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.