I got this error in my program which grab data from different website and write them to a file:
'charmap' codec can't encode characters in position 151618-151624: character maps to <undefined>
I am not familiar with all the encoding decoding thing and I have been okay with what python 2 did. Although the python officially said that they made the change in order to make things better, it seems to get worse.
I have no idea how to fix these errors. However I am a pro-active person so I would really like to know what is causing the problem and how to solve it. I have check the official site but the words are hard to understand.
Could I have a simple elaboration on that? Another page is also acceptable.
EDIT: I've check this page, the Unicode HOWTO in Python 2.7. My understanding is that we must translate the unicode string into binary format while we're writing it to file and it require an encoding. Obviously 'utf-8' is the best one, but why it didn't force the python interpreter to use 'utf-8'? Instead, it use some strange codec such as 'charmap' and 'cp950' and 'latin-1'.