Converting string from MongoDB into int using MongoDB decode

Question

I am having some trouble with the mongodb driver for Go when trying to decode a property in my collection which is of type string to a property in my struct which is of type int. I have a collection that has a property that is supposed to be of type int that was converted to type string by someone editing a single document. Since all of the documents have that property as int except that one, I am getting an error "cannot decode string into an integer type". I figured out a way to do it but it's not something that "should" be done. I modified the IntDecodeValue func in the default_value_decoders.go in the mongodb driver for Go. Below is what I added.

    case bsontype.String:
    s, err := vr.ReadString()
    if err != nil {
        return err
    }
    i32, err := strconv.Atoi(s)
    if err != nil {
        return err
    }
    i64 = int64(i32)

I know this will be overwritten when I update the driver but have been beating my head against the wall trying to figure out how to handle such a case. I know the best solution is to not allow editing of the documents directly but want to account for that case.

The only viable solution is to remove access for anyone foolish enough to hand-edit documents in the database. There's no way you can account for the literally infinite ways a user can mangle data when modifying it by hand. What if they later change a string to an int? Or an array to a string? You can't just keep adding hacks to the Mongo driver to account for the misdeeds of people who clearly shouldn't have DB access in the first place. — Adrian
– Adrian, Commented Feb 13, 2020 at 16:52
I totally agree with that assessment. I was just hoping there was a way that I could extend that function to be able to handle said mangling gracefully without the app barfing or excluding the document that has been mangled. — Don Ellis
– Don Ellis, Commented Feb 13, 2020 at 17:05
Yes, sort of. You'll need to add another hack for every way that they might mangle the data that you want to be able to handle gracefully. These will pile up, and make it harder to keep up to date with security patches. That said... your question doesn't actually have a question in it. What is your question exactly? — Adrian
– Adrian, Commented Feb 13, 2020 at 17:22
Is there a way in go to extend that function so I can add at least some of the ways it might become mangled. I have only been learning go for a couple of weeks and afaik there is no extending functions as there is in other languages. — Don Ellis
– Don Ellis, Commented Feb 13, 2020 at 17:24
I decided that it's probably not a path I want to go down as Adrian said, too many possibilities for mangling. I am going to set up some validation rules on the collection as I create them so IF someone does manually edit a document, they cannot violate the datatypes on the collection — Don Ellis
– Don Ellis, Commented Feb 13, 2020 at 21:10

Divjot Arora · Accepted Answer · 2020-02-15 00:59:26Z

2

You can do this via a custom BSON decoder. An example of this in the documentation is https://pkg.go.dev/go.mongodb.org/mongo-driver/bson/bsoncodec?tab=doc#example-Registry-CustomDecoder. For your specific use case, the following should work:

func intDecodeValue(dc bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) {

// Same logic as DefaultValueDecoders.IntDecodeValue
// In the switch statement for vr.Type, you can add a case for bsontype.String

}

To register this decoder so it's used when performing CRUD operations, you can use the SetRegistry client option:

decoder := bsoncodec.ValueDecoderFunc(intDecodeValue)
registry := bson.NewRegistryBuilder().RegisterDefaultDecoder(reflect.Int, decoder).Build()

clientOpts := options.Client().SetRegistry(registry)
client, err := mongo.Connect(context.TODO(), clientOpts)

Note that because Go differentiates between different integer types (e.g. int8/int16/int32/int64), you'll need to call RegisterDefaultDecoder to register the custom decoder for each integer type that you may see in your structs, similar to what is done in the RegisterDefaultDecoders function in default_value_decoders.go.

answered Feb 15, 2020 at 0:59

Divjot Arora

522 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Don Ellis Over a year ago

Yep, that's what I ended up doing until I decided that I'm not even going to do a check to make sure each document has the same data structure. I am using validation rules on the collection itself that prevent the changing of datatypes altogether. Thanks for all your help folks.

SPM · Accepted Answer · 2020-02-13 18:05:11Z

Here's a possible solution to decode that string field as int in your struct handling both string and int fields.

I used the following struct for this example:

type Obj struct {
    Field1   string `bson:"field1"`
    IntField int    `bson:"intField"`
}

And inserted the following documents (note the second doc with the string field as "intField": "2"):

db.s2i.insertOne({"field1": "value", "intField": 3})
db.s2i.insertOne({"field1": "value2", "intField": "2"})

Using the $toInt operator from the aggregation framework like this:

pipeline := []bson.M{bson.M{"$match": bson.M{}}, bson.M{"$project": bson.M{"field1": 1, "intField": bson.M{"$toInt": "$intField"}}}}

cur, err := client.Database("stack").Collection("s2i").Aggregate(context.TODO(), pipeline)

for cur.Next(context.TODO()) {
    var res *Obj
    err = cur.Decode(&res)

    fmt.Println(res, err)
    fmt.Println("Value of res.IntField:")
    fmt.Println(res.IntField)
    fmt.Println("Type of res.IntField:")
    fmt.Println(reflect.TypeOf(res.IntField))
}

Returns the following documents, with the "2" decoded as an int:

&{value 3} <nil>
Value of res.IntField:
3
Type of res.IntField:
int
&{value2 2} <nil>
Value of res.IntField:
2
Type of res.IntField:
int

Collectives™ on Stack Overflow

Converting string from MongoDB into int using MongoDB decode

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related