python - UnicodeEncodeError: 'latin-1' codec can't encode character

Question

What could be causing this error when I try to insert a foreign character into the database?

>>UnicodeEncodeError: 'latin-1' codec can't encode character u'u201c' in position 0: ordinal not in range(256)

And how do I resolve it?

Thanks!

Answer 1

I ran into this same issue when using the Python MySQLdb module. Since MySQL will let you store just about any binary data you want in a text field regardless of character set, I found my solution here:

Using UTF8 with Python MySQLdb

Edit: Quote from the above URL to satisfy the request in the first comment...

"UnicodeEncodeError:'latin-1' codec can't encode character ..."

This is because MySQLdb normally tries to encode everythin to latin-1. This can be fixed by executing the following commands right after you've etablished the connection:

db.set_character_set('utf8')
dbc.execute('SET NAMES utf8;')
dbc.execute('SET CHARACTER SET utf8;')
dbc.execute('SET character_set_connection=utf8;')

"db" is the result of MySQLdb.connect(), and "dbc" is the result of db.cursor().