76

Is there a reliable way to JSON.stringify a JavaScript object that guarantees that the ceated JSON string is the same across all browsers, Node.js and so on, given that the JavaScript object is the same?

I want to hash JavaScript objects like

{
  signed_data: object_to_sign,
  signature:   md5(JSON.stringify(object_to_sign) + secret_code)
}

and pass them around across web applications (e.g. Python and Node.js) and the user so that the user can authenticate against one service and show the next service "signed data" for that one to check if the data is authentic.

However, I came across the problem that JSON.stringify is not really unique across the implementations:

  • In Node.js / V8, JSON.stringify returns a JSON string without unnecessary whitespace, such as '{"user_id":3}.
  • Python's simplejson.dumps leaves some whitespace, e.g. '{"user_id": 3}'
  • Probably other stringify implementations might deal differently with whitespace, the order of attributes, or whatever.

Is there a reliable cross-platform stringify method? Is there a "nomalised JSON"?

Would you recommend other ways to hash objects like this?

UPDATE:

This is what I use as a workaround:

normalised_json_data = JSON.stringify(object_to_sign)
{
  signed_data: normalised_json_data,
  signature:   md5(normalised_json_data + secret_code)
}

So in this approach, not the object itself, but its JSON representation (which is specific to the sigining platform) is signed. This works well because what I sign now is an unambiguous string and I can easily JSON.parse the data after I have checked the signature hash.

The drawback here is that if I send the whole {signed_data, signature} object as JSON around as well, I have to call JSON.parse twice and it does not look as nice because the inner one gets escaped:

{"signature": "1c3763890298f5711c8b2ea4eb4c8833", "signed_data": "{\"user_id\":5}"}
Improve this question
3
  • 2
    you're using json stringify - which as a serialization mechanism - to do hashing. i'm not sure that's a good idea - precisely for the reasons you've encountered. regardless, JSON.stringify is very limited, and i wouldn't trust it to hash or serialize my information. for example, try JSON.stringify(new Error('not going to work')) Commented May 14, 2015 at 9:17
  • I also want to self-comment that MD5 isn't the best hash function to use here. Commented May 14, 2015 at 23:07
  • 3
    JSON.stringify is not a good way to prepare for hashing i.e JSON.stringify({b:2,a:1}) => '{"b":2,"a":1}' , while JSON.stringify({a:1,b:2}) => '{"a":1,"b":2}' Commented Mar 8, 2019 at 22:56

6 Answers 6

Reset to default
94

You might be interested in npm package object-hash, which seems to have a rather good activity & reliability level.

var hash = require('object-hash');

var testobj1 = {a: 1, b: 2};
var testobj2 = {b: 2, a: 1};
var testobj3 = {b: 2, a: "1"};

console.log(hash(testobj1)); // 214e9967a58b9eb94f4348d001233ab1b8b67a17
console.log(hash(testobj2)); // 214e9967a58b9eb94f4348d001233ab1b8b67a17
console.log(hash(testobj3)); // 4a575d3a96675c37ddcebabd8a1fea40bc19e862
Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Or for in browser use, console.log(objectHash(testobj1)).
That's very much not cross platform, though, which is what the OP asked for. There are also deterministic versions wrappers around JSON that run much faster than object-hash. See this benchmark I made: jsperf.com/stringify-vs-object-hash
@oligofren npmjs.com/package/json-stringify-deterministic looks interesting. I would advise to post your comment as another answer. Concerning cross platform support, I agree with Mark Kahn answer... + maybe making that JS hashing function a webservice, then calling it from the different platforms would do the trick without too much dev cost.
I tested json-stringify-deterministic, but found it was quite slow. A simple function of my own reduced the total time from 4 ms to 2 ms for hashing: function ObjectToUniqueStringNoWhiteSpace(obj : any){ let SortedObject : any = sortObjectKeys(obj); let jsonstring = JSON.stringify(SortedObject, function(k, v) { return v === undefined ? "undef" : v; }); // Remove all whitespace let jsonstringNoWhitespace :string = jsonstring.replace(/\s+/g, ''); return jsonstringNoWhitespace; }
Please note that SHA1 and MD5 used by this library are NOT cryptographically secure. So use carefully and not for cryptographic purposes.
10

This is an old question, but I thought I'd add a current solution to this question for any google referees.

The best way to sign and hash JSON objects now is to use JSON Web Tokens. This allows for an object to be signed, hashed and then verified by others based on the signature. It's offered for a bunch of different technologies and has an active development group.

Improve this answer

5 Comments

Tried to generate jwt signature, but looks like it does not ignore field order in payload json. So I do not understand how it can help since OP wanted to ignore field order during hash calculation. Could you please explain your idea?
@derp Javascript has never guaranteed the order of keys in objects - this is not something that you should consider when using objects. I don't think the OP said specifically that he wanted to ignore field order, only that he thought that different implementations would handle them differently. His work around effectively handles this because he is signing the end-result, not another stringify call.
He mentioned in item number three of stringify disadvantages that resulting string can have different fields order. I still do not understand how JWT can produce the same hash/signature for {"foo":"bar", "bar":"foo"} and {"bar":"foo","foo":"bar"} which is basically the same except their fields order. If there is no such for JWT, your answer is not the exact thing the OP (and me) was looking for.
Don't confuse signatures and hashes, they solve different problems. A signature is like a hash on steroids with a focus on security. Hashes have nothing to do with security. JWT is probably the best solution for the OPs question as you get secure hashing, verification, expiry and trust included automatically. You will get different signatures for the same object every time because a timestamp is included in the hashed data automatically. It sounds like @Derp just wants identical hashes between logically identical object representations, in which case see object-hash
@MattForster Actually, as of ES6, object key order is guaranteed by the ECMAScript spec.
10

After trying some hash algorithms and JSON-to-string methods, I found this to work the best (Sorry, it is typescript, can of course be rewritten to javascript):

// From: https://stackoverflow.com/questions/5467129/sort-javascript-object-by-key
function sortObjectKeys(obj){
    if(obj == null || obj == undefined){
        return obj;
    }
    if(typeof obj != 'object'){ // it is a primitive: number/string (in an array)
        return obj;
    }
    return Object.keys(obj).sort().reduce((acc,key)=>{
        if (Array.isArray(obj[key])){
            acc[key]=obj[key].map(sortObjectKeys);
        }
        else if (typeof obj[key] === 'object'){
            acc[key]=sortObjectKeys(obj[key]);
        }
        else{
            acc[key]=obj[key];
        }
        return acc;
    },{});
}
let xxhash64_ObjectToUniqueStringNoWhiteSpace = function(Obj : any)
{
    let SortedObject : any = sortObjectKeys(Obj);
    let jsonstring = JSON.stringify(SortedObject, function(k, v) { return v === undefined ? "undef" : v; });

    // Remove all whitespace
    let jsonstringNoWhitespace :string = jsonstring.replace(/\s+/g, '');

    let JSONBuffer: Buffer = Buffer.from(jsonstringNoWhitespace,'binary');   // encoding: encoding to use, optional.  Default is 'utf8'
    return xxhash.hash64(JSONBuffer, 0xCAFEBABE, "hex");
}

It used npm module: https://cyan4973.github.io/xxHash/ , https://www.npmjs.com/package/xxhash

The benefits:

  • This is deterministic
  • Ignores key order (preserves array order)
  • Cross platform (if you can find equivalents for JSON-stringify) JSON-stringify will hopefully will not get a different implementation and the whitespace removal will hopefully make it JSON-formatting independent.
  • 64-bit
  • Hexadecimal string a result
  • Fastest (0.021 ms for 2177 B JSON, 2.64 ms for 150 kB JSON)
Improve this answer

Comments

9

You're asking for an implementation of something across multiple languages to be the same... you're almost certainly out of luck. You have two options:

  • check www.json.org implementations to see if they might be more standardized
  • roll your own in each language (use json.org implementations as a base and there should be VERY little work to do)
Improve this answer

1 Comment

OK, for critical things like signing authentication, it looks that I'm indeed out of luck with this approach. I updated my post to show how I sign not the object, but the specific JSON string of the hash-creating platform.
7

You could normalise the result of stringify() by applying rules such as:

  • remove unnecessary whitespace
  • sort attribute names in hashes
  • well-defined consistent quoting style
  • normalise string contents (so "\u0041" and "A" become the same)

This would leave you with a canonical JSON representation of your object, which you can then reliably hash.

Improve this answer

8 Comments

quoting style is normalized (it's part of the JSON spec). White-space and attribute order aren't.
@cwolves: That's true, but I've seen single-quoted strings that purport to be JSON.
Perhaps, but those are broken and I really doubt you'd see that in any JSON implementation native to a language.
How do you want to sort keys in a JS object?
@YanFoto You need to define that yourself. Could be alphanumeric, for instance. Or you could rely on a package such as json-stringify-deterministic. Benchmark: jsperf.com/stringify-vs-object-hash
|
1

You may find bencode suitable for your needs. It's cross-platform, and the encoding is guaranteed to be the same from every implementation.

The downside is it doesn't support nulls or booleans. But that may be okay for you if you do something like transforming e.g., bools -> 0|1 and nulls -> "null" before encoding.

Improve this answer

1 Comment

Also float is not supported

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.