2

I'm building a json web service which contains strings with characters like é,ñ,Á,etc.

Using python I found this snippet of code that works perfectly when I run it in console:

import json
string = "NIÑO, ÁRBOL, HÉROE"
print json_dumps({'string': string}, ensure_ascii=False, encoding='utf-8')

The thing is that I'm using Django, and it doesn't look like it is as straightforward as the code above. Here's a peek of what I'm doing at my views.py file

import json
from models import *
from django.http import HttpResponse
from django.db.models import Q

def jsonService(request):
    # The following line performs a query at the db
    myObjects = MyObjects.objects().filter( Q(...) ) 

    result = {"string": myObject[0].field } # let's say myObject[0].field contains "NIÑO, ÁRBOL, HÉROE"

    # Prepares a response
    response = HttpResponse(json.dumps(result, ensure_ascii=False, encoding="utf-8"))
    # Sets the heades of content type and its charset
    response['Content-type'] = 'application/json; charset=utf-8'

    # Returns the response we prepared
    return response

The output of this code is:

{ 
    string : "NIôO, ÃRBOL, HÃ%ROE"
}

if I apply python's function repr() to the string myObject[0].field when I assemble the result object, to my surprise the result is:

{ 
    string : "NI\xc3\u2018O, \xc3\x81RBOL, H\xc3\u2030ROE"
}

What I can infere from here is that maybe the strings that the db delivers (which are unicode strings acording to python's type()) are encoded in a format other that utf-8, but it thows me the following error:

'ascii' codec can't decode byte 0xc3 in position 14: ordinal not in range

Those escaped characters seems very strange to me, not to mention that I don't want unicode strings but the accented characters (something like {string: "NIÑO, ÁRBOL, HÉROE"}), I know it's possible because I've seen some google services work with accents.

Some advise? maybe I'm doing something incredibly wrong that I haven't realized, that's why I described the full process.

6
  • Have you confirmed that the database is using utf-8? mySQL does not by default. Commented Dec 17, 2012 at 2:37
  • @AgDude I'm not sure how is that. I filled the database with a python script, where i specified when opening the connection the encoding utf-8, and when I uploaded the db to my server, heroku btw, I also specified the same encoding. Will it do or I missed to apply some special config to my dbm? Commented Dec 17, 2012 at 7:04
  • I have not used heroku, but check out this answer. As always, if you are altering an existing database, make a backup first! Commented Dec 17, 2012 at 11:41
  • @AgDude Before modifying heroku's configuration, I query for the current on with heroku config and the default value was already en_US.UTF-8, I changed to es_ES.UTF-8 and server restarted automatically, but nothing. I've been researching about this `\xc3` character, maybe it is the remanent of some previous encoding, but not sure yet. Commented Dec 18, 2012 at 4:45
  • 1
    It looks like it is coming out of the database already incorrect. Use a pdb.set_trace() to view the object in the python shell using runserver. How was the data inserted into the database? Can you look at the data directly in a SQL client? Commented Dec 18, 2012 at 16:44

1 Answer 1

3

Try this in your views.py works fine for me

from django.utils import simplejson
from django.core.serializers.json import DjangoJSONEncoder

def jsonService(request):

    myObjects = MyObjects.objects().filter( Q(...) )
    fields = [x.field for x in myObjects] # creates a list of all fileds 

    # it will create JSON object
    result=simplejson.dumps({
        'fileds':fileds,
     })

    return HttpResponse(result, mimetype='application/json')
Sign up to request clarification or add additional context in comments.

1 Comment

if I use this method I get the same result as when using json {string : "NIôO, ÃRBOL, HÃ%ROE"}. However I'm afraid that maybe there's something missing, because DjangoJSONEncoder it's not used, is it? I'm very optimistic about this approach because a workmate advised me to use simplejson too.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.