Encoding issue using Microsoft Graph API and ArduinoJSON for parsing

Question

I am building a door sign for our Company's rooms which connects to a shared Office 365 calendar.

The string "Känel" results in "Ku00e4nel" when I parse the response of the Graph API using ArduinoJSON which doesn't support either UTF-8 nor UTF-16 but "u00e4" is Unicode so what is going on here?

The developer told me that parsed strings aren't touched so the output should work if the source is in UTF-8 which regarding the output apparently is not.

I have already tried setting the charset in the header but maybe I am not aware of further available options which would help me to get rid of my issue.

Code:

String response = "";
DynamicJsonBuffer JSONBuffer;

http.begin(graphAPI);
http.addHeader("Authorization", "Bearer " + token);
http.addHeader("Prefer", "outlook.timezone = \"Central Europe Standard Time\"");
http.addHeader("Content-type", "application/json; charset=UTF-8");

int httpResponseCode = http.GET();

if (httpResponseCode = 200) {
    response = http.getString();                       
    JsonObject& parsed = JSONBuffer.parseObject(response);

    if (parsed.success()) {
        for (int i = 0; i < parsed["value"].size(); i++) {
            String startTime = parsed["value"][i]["start"]["dateTime"];
            startTime = startTime.substring(11, 16);
            String endTime = parsed["value"][i]["end"]["dateTime"];
            endTime = endTime.substring(11, 16);
            String subject = parsed["value"][i]["subject"];

            next_y += 35;
            display_u8.setCursor(static_x, next_y);

            display_u8.print(startTime + " - " + endTime + " " + subject);
        }
    } else {
        display_u8.println("Parsing failed");
    }

Sample Response:

"value": [
{
    "@odata.etag": "W/"Npd9zqlLQE+/S5XEMoC9AAAAIkfORw=="",
    "id": ".............",
    "subject": "von Känel Sacha ",
    "start": {
        "dateTime": "2019-02-11T13:00:00.0000000",
        "timeZone": "Central Europe Standard Time"
    },
    "end": {
        "dateTime": "2019-02-11T14:00:00.0000000",
        "timeZone": "Central Europe Standard Time"
    }
},
{
    "@odata.etag": "W/"Npd9zqlLQE+/S5XEMoC9AAAAIkfOiQ=="",
    "id": ".............",
    "subject": "von Känel Sacha ",
    "start": {
        "dateTime": "2019-02-11T15:00:00.0000000",
         "timeZone": "Central Europe Standard Time"
    },
    "end": {
        "dateTime": "2019-02-11T16:00:00.0000000",
        "timeZone": "Central Europe Standard Time"
    }
}
],
}

The parsed strings should be displayed as "von Känel Sacha" and not as "von Ku00e4nel Sacha".

Edit: I just discovered that the source of the problem is the Microsoft Graph API itself which encodes the character "ä" as \u00e4 but that's wrong in my opinion keeping in mind I set the charset to UTF-8 which will not work with that Unicode representation.

pewpew · Accepted Answer · 2019-02-11 14:12:30Z

1

According to the JSON specification, the escaping of such characters in UNICODE is to achieve full coverage of characters used in the response and therefore the source of the problem is not an issue with Microsofts Graph API but the Arduino JSON library itself which is unable to meet the JSON specifications.

As a solution, I now used this ArduinoJSON fork which does work flawlessly https://github.com/trilader/ArduinoJson

answered Feb 11, 2019 at 14:12

pewpew

335 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Will Huang Over a year ago

I think Microsoft should offer an option to not escape these Unicode characters.

Collectives™ on Stack Overflow

Encoding issue using Microsoft Graph API and ArduinoJSON for parsing

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related