14

I am trying to add BigInt support within my library, and ran into an issue with JSON.stringify.

The nature of the library permits not to worry about type ambiguity and de-serialization, as everything that's serialized goes into the server, and never needs any de-serialization.

I initially came up with the following simplified approach, just to counteract Node.js throwing TypeError: Do not know how to serialize a BigInt at me:

// Does JSON.stringify, with support for BigInt:
function toJson(data) {
    return JSON.stringify(data, (_, v) => typeof v === 'bigint' ? v.toString() : v);
}

But since it converts each BigInt into a string, each value ends up wrapped into double quotes.

Is there any work-around, perhaps some trick within Node.js formatting utilities, to produce a result from JSON.stringify where each BigInt would be formatted as an open value? This is what PostgreSQL understands and supports, and so I'm looking for a way to generate JSON with BigInt that's compliant with PostgreSQL.

Example

const obj = {
    value: 123n
};

console.log(toJson(obj));

// This is what I'm getting: {"value":"123"}
// This is what I want: {"value":123}

Obviously, I cannot just convert BigInt into number, as I would be losing information then. And rewriting the entire JSON.stringify for this probably would be too complicated.

UPDATE

At this point I have reviewed and played with several polyfills, like these ones:

But they all seem like an awkward solution, to bring in so much code, and then modify for BigInt support. I am hoping to find something more elegant.

Improve this question
3
  • 1
    I don't see a way to do this with native JSON.stringify either (apart from lobbying for better native support). I'm pretty certain though that you can simplify the code of the polyfills a lot by stripping unnecessary features (indentation, ES3 compatibility). Commented Oct 5, 2019 at 19:02
  • 1
    @Bergi I've been drawing toward the same conclusion, but still hoping to be wrong, to avoid messing with polyfills modifications. With this thing being so generic, it would be asking for its own package, methinks. Commented Oct 5, 2019 at 19:04
  • @Bergi See my own answer, after all the "impossible" feedback I had :))) Commented Oct 6, 2019 at 0:17

3 Answers 3

Reset to default
6

Solution that I ended up with...

Inject full 123n numbers, and then un-quote those with the help of RegEx:

function toJson(data) {
    return JSON.stringify(data, (_, v) => typeof v === 'bigint' ? `${v}n` : v)
        .replace(/"(-?\d+)n"/g, (_, a) => a);
}

It does exactly what's needed, and it is fast. The only downside is that if you have in your data a value set to a 123n-like string, it will become an open number, but you can easily obfuscate it above, into something like ${^123^}, or 123-bigint, the algorithm allows it easily.

As per the question, the operation is not meant to be reversible, so if you use JSON.parse on the result, those will be number-s, losing anything that's between 2^53 and 2^64 - 1, as expected.

Whoever said it was impossible - huh? :)

UPDATE-1

For compatibility with JSON.stringify, undefined must result in undefined. And within the actual pg-promise implementation I am now using "123#bigint" pattern, to make an accidental match way less likely.

And so here's the final code from there:

 function toJson(data) {
    if (data !== undefined) {
        return JSON.stringify(data, (_, v) => typeof v === 'bigint' ? `${v}#bigint` : v)
            .replace(/"(-?\d+)#bigint"/g, (_, a) => a);
    }
}

UPDATE-2

Going through the comments below, you can make it safe, by counting the number of replacements to match that of BigInt injections, and throwing error when there is a mismatch:

function toJson(data) {
    if (data !== undefined) {
        let intCount = 0, repCount = 0;
        const json = JSON.stringify(data, (_, v) => {
            if (typeof v === 'bigint') {
                intCount++;
                return `${v}#bigint`;
            }
            return v;
        });
        const res = json.replace(/"(-?\d+)#bigint"/g, (_, a) => {
            repCount++;
            return a;
        });
        if (repCount > intCount) {
            // You have a string somewhere that looks like "123#bigint";
            throw new Error(`BigInt serialization conflict with a string value.`);
        }
        return res;
    }
}

though I personally think it is an overkill, and the approach within UPDATE-1 is quite good enough.

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

I would recommend to take @Melchia's approach though and have the toJson callback return an object like {"$type":"bigint","value":"…"}, then match that with your regex. This should make an accidental match almost impossible. It is far too likely that the JSON contains user-controlled arbitrary string data, which can get handcrafted for an injection, while it is rather unusual that the complete object is user-controlled.
@Bergi Actually, his suggestion was into nowhere, as it resolved nothing. He didn't suggest any regex use at all. And now that I am using "123#bigint", it is unique enough not to get mixed with a random string. I see no benefit in complicating it by converting into a object. It would end up as the same string, just more convoluted for regex to parse.
The difference is that I consider that the …#bigint pattern is not unique enough. Maybe it's unlikely to come up randomly, but it seems easy to inject for an attacker who controls some string values in the object. The regex would be longer, yes, but not more convoluted.
As an extra security measure, you add a counter for how many bigints you meet in the object, and another counter for how often the replace callback is applied, then throw an exception if they don't match rather than storing invalid data in the database.
@Bergi Throwing an error on replacement count mismatch is actually a good idea. I was considering it myself earlier.
|
1

If you are using Typescript on express then place the following code on the main server file. Easy Hack 😎 works fine

BigInt.prototype['toJSON'] = function () {
    return parseInt(this.toString());
};
Improve this answer

3 Comments

This hack will kill everything above 53 bits, so it's no good.
Can you give me more insight or referred me to the issue? Thanks
The insight is right in my answer (the accepted one). Int can only hold up to 53 bits of a 64-bit number in JavaScript.
-2

If there is no problem using it as number, you may convert it:

let TheBigInt = BigInt(10);
let TheNumber = Number(TheBigInt);
Improve this answer

1 Comment

The question is about automatic conversion, for an entire object. Manual conversion is of no use here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.