UTF8 character encoding using node Request module

Question

I have a simple app that uses a jquery ajax request to send form data to a node server, which in turn submits to a third party api using the Request module for node js.

The issue I'm having is that accented (and other similar) characters are not encoded correctly when they reach the third party server. For example é is recorded as Ã©

I am fairly sure this is to do with the settings for Request as I get the same results when I bypass the ajax call.

Here are the settings I am using:

html:

<meta http-equiv="Content-type" content="text/html; charset=utf-8" />

jquery ajax settings:

type        : 'POST',
url         : '/api',
data        : formData, // A json object
dataType    : 'json',
ContentType : 'text/html; charset=utf-8'

Request module settings in node (there is nothing happening to the form data between ajax post and being sent by request):

request.post({
    url: "https://testurl.com/api/",
    form: formData,
    headers: {'Content-Type': 'application/json; charset=utf-8'}
} ...

I have read various SO solutions but had no success, so any suggestions greatly appreciated.

Have you tried setting the "encoding" - option of request specifically to "utf-8"? Do you have a body-parser in your application stack? (so does the formData actually get parsed as JSON before you assign it to request or is it still a string there?). What happens when you console.log(formData) on your Node-Server? — David Losert
– David Losert, Commented Mar 12, 2015 at 8:59
Yes I've tried encoding: 'utf-8', same result. Im using Body Parser: app.use(bodyParser.urlencoded({ extended: false })); and app.use(bodyParser.json()); When i log formData just before it goes to request I get what looks like a javascript object { first_name: 'é' } — gethyn1
– gethyn1, Commented Mar 12, 2015 at 9:47
You said you get "the same results" when you bypass the ajax call? You mean sending the data to the third party directly? Then it works? — David Losert
– David Losert, Commented Mar 12, 2015 at 11:35
I mean when I use the Request post route on the node server without involving ajax, I still get the incorrect character. If I post directly to the third party I don't get the error (using their endpoint without node or ajax involved). This suggests to me that the issue is with Request. — gethyn1
– gethyn1, Commented Mar 12, 2015 at 13:24
Try to set encoding: 'utf8' (without the Dash) - according to the thread here it cold be the solution: stackoverflow.com/questions/8332500/… — David Losert
– David Losert, Commented Mar 12, 2015 at 13:51

gethyn1 · Accepted Answer · 2015-06-01 09:57:07Z

2

After researching character encoding I discovered that the issue is to do with multibyte encoding of various characters (a quick google will find some good SO posts on the subject).

I thought it was odd that Request didn't handle this automatically, so I played around with the Request syntax and managed to fix the issue. Here is my revised code that works:

request(
    {
        url: 'https://testurl.com/api/',
        method: 'POST',
        json: true,
        headers: {
            'content-type': 'application/json'
        },
        body: formData
    },
    function (error, response, body) {
        ...     
    }
);

answered Jun 1, 2015 at 9:57

gethyn1

1371 gold badge1 silver badge9 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

UTF8 character encoding using node Request module

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related