Update Nested fields using Golang struct

Question

I am facing a issue with update struct fields using golang

As the var name suggest some current config fields should be updated with the requested config

     currentConfig:=`
     {
         "field_one": "value",
         "data": {
             "field_two": [
                 "data1",
                 "data2"
             ],
             "field_three": "check",
             "field_four": 12
         },
        "field_five": [
                 "data1",
                 "data2"
             ],
        "data2": {
            "field_six":{
                "field_seven": 100
            }
        }
     }`
    
     
     updateRequest:=`
     {
         "data": {
             "field_three": "updated check" //ignore if same value exist (field_three exists in current config)
         },
        "field_five": ["data3"],    // append to current config
        "data2": {
            "field_six":{
                "field_eight": 300  // add value if doesnt exist to current
            }
        }
     }`

func main() {
    config := make(map[string]interface{})
    err := json.Unmarshal([]byte(currentConfig), &config)
    if err != nil {
        panic(err)
    }
    updateFields := make(map[string]interface{})
    err1 := json.Unmarshal([]byte(updateRequest), &updateFields)
    if err1 != nil {
        panic(err1)
    } 

    fmt.Println(config)
    updateFields = ParseJsonMap(updateFields, config)
    fmt.Println(updateFields)
}


func ParseJsonMap(aMap map[string]interface{}, finalMap map[string]interface{}) map[string]interface{} {
    parseMap("", aMap, &finalMap)
    return finalMap
}

Traverses the struct and updates the fields

func parseMap(k string, aMap map[string]interface{}, finalMap *map[string]interface{}) {
    if len(aMap) == 0 {
        (*finalMap)[k] = nil
        return
    }

    for key, val := range aMap {
        if val != nil {
            switch concreteVal := val.(type) {
            case map[string]interface{}:
                if _, ok := (*finalMap)[getKey(k, key)]; ok {
                    parseMap(getKey(k, key), val.(map[string]interface{}), finalMap)
                } else {
                    (*finalMap)[getKey(k, key)] = val
                }
            case []interface{}:
                res := val.([]interface{})
                if arr, ok := (*finalMap)[getKey(k, key)]; ok {
                    for _, valueIn := range res {
                        arr = append(arr.([]interface{}), valueIn)
                    }
                    (*finalMap)[getKey(k, key)] = arr
                } else {
                    (*finalMap)[getKey(k, key)] = res
                }
                
            default:
                concreteValType := reflect.TypeOf(concreteVal)
                if concreteValType.Kind() == reflect.Map {
                    parseMap(getKey(k, key), concreteVal.(map[string]interface{}), finalMap)
                } else {
                    if _, ok := (*finalMap)[getKey(k, key)]; !ok {
                        (*finalMap)[getKey(k, key)] = concreteVal
                    }

                }
            }
        } else {
            (*finalMap)[getKey(k, key)] = nil
        }
    }
}

func getKey(k string, key string) string {
    if k == "" {
        return key
    }

    return k + "." + key
}

Expected Result

map[data:map[field_four:12 field_three:check field_two:[data1 data2]] data2:map[field_six:map[field_eight:300 field_seven:100]] field_five:[data1 data2 data3] field_one:value]

{
         "field_one": "value",
         "data": {
             "field_two": [
                 "data1",
                 "data2"
             ],
             "field_three": "check",  //since key exist with data not updated
             "field_four": 12
         },
        "field_five": [
                 "data1",
                 "data2",
                 "data3"   //data 3 appended
             ],
        "data2": {
            "field_six":{
                "field_seven": 100,
                "field_eight": 300 //field is added
            }
        }
     }

Result got - created key at top level

map[data:map[field_four:12 field_three:check field_two:[data1 data2]] data.field_three:check changed data2:map[field_six:map[field_seven:100]] data2.field_six:map[field_eight:300] field_five:[data1 data2 data3] field_one:value]

Just want to know if this is supported, if yes can you help me with it and if any better approaches exist

You seem to be attaching some significance to the . used in getKeys but json spec does not. If you want to change field_three under data or whatever, you need to do it within that map, not within the parent map with a dot in the key. That's JSON's rule. — erik258
– erik258, Commented Oct 6, 2021 at 16:57
Correct. I was looking into implementing that have no idea as to how to achieve that. Some help would be appreciated — user17090811
– user17090811, Commented Oct 6, 2021 at 17:03
Note that Go does have actual struct types but in your code there is not a single struct there. You might want to be more precise when describing the problem for which you would like a solution. — mkopriva
– mkopriva, Commented Oct 6, 2021 at 17:08
Generally I'm a big fan of modeling data in structs, but in this case structs don't help much, since the keys are not known. Using interface{}, []interface{}, and map[string]interface{} with type checks makes sense to me — erik258
– erik258, Commented Oct 6, 2021 at 17:16

colm.anseo · Accepted Answer · 2021-10-06 18:31:04Z

4

It appears you're trying to overlay one map onto another map. This gets complicated if you don't want to apply this kind of update without altering an existing map. So it may be easier to separate the two steps:

Copy a map[string]interface{}
overlay one map[string]interface{} on top of another

func CopyMap(m map[string]interface{}) map[string]interface{} {
    cp := make(map[string]interface{})
    for k, v := range m {
        vm, ok := v.(map[string]interface{})
        if ok {
            cp[k] = CopyMap(vm)
        } else {
            cp[k] = v
        }
    }
    return cp
}

func overlay(dst, src map[string]interface{}) error {
    for k, v := range src {

        if _, ok := dst[k]; !ok {
            dst[k] = v // easy case - dst key does not exist
            continue
        }

        d, ok1 := dst[k].(map[string]interface{})
        s, ok2 := src[k].(map[string]interface{})

        if ok1 && ok2 {
            overlay(d, s) // merge case
        } else if !ok1 && !ok2 {
            dst[k] = v // non-map - so simple assignment/reassignment
        } else {
            return fmt.Errorf("incompatible update types") // map to non-map or vice-versa
        }

    }
    return nil
}

to use:

err := json.Unmarshal([]byte(currentConfig), &config) // check err
err = json.Unmarshal([]byte(updateRequest), &updateFields) // check err
newconfig = CopyMap(config)
err = overlay(newconfig, updateFields) // check err

https://play.golang.org/p/RZPbkv19ChL

Output:

    config : map[data:map[field_four:12 field_three:check field_two:[data1 data2]] data2:map[field_six:map[field_eight:200 field_seven:100]] field_five:[data1 data2] field_one:value]
    update : map[data:map[field_three:check changed] data2:map[field_six:map[field_eight:300]] field_five:[data1]]
 newconfig : map[data:map[field_four:12 field_three:check changed field_two:[data1 data2]] data2:map[field_six:map[field_eight:300 field_seven:100]] field_five:[data1] field_one:value]

UPDATE: to handle appends rather than replacements for JSON arrays:

func overlay2(dst, src map[string]interface{}) error {
    for k, v := range src {

        if _, ok := dst[k]; !ok {
            dst[k] = v // easy case - dst key does not exist
            continue
        }

        dm, ok1 := dst[k].(map[string]interface{})
        sm, ok2 := src[k].(map[string]interface{})

        if ok1 && ok2 {
            overlay2(dm, sm) // merge case
            continue
        }

        ds, ok1 := dst[k].([]interface{})
        ss, ok2 := src[k].([]interface{})

        if ok1 && ok2 {
            dst[k] = append(ds, ss...) // JSON array case
            continue
        }

        return fmt.Errorf("unhandled type/update")

    }
    return nil
}

https://play.golang.org/p/i-0yXMcqU7Z

edited Oct 6, 2021 at 18:31

answered Oct 6, 2021 at 17:13

colm.anseo

22.7k5 gold badges50 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

user17090811 Over a year ago

thank you @colm.anseo for your effort. i forgot to add that the config should be updated based on certain conditions. have updated the question with comments in config.

erik258 Over a year ago

this is close, except it doesn't quite handle "field_five":["data3"]. According to desired results it should append to existing field_five, not replace entire array.

colm.anseo Over a year ago

@DanielFarrell that requirement wasn't in the original question. Updating by append seems like a bad approach. How would one make an update to delete elements from the array? It would not be possible with this approach.

colm.anseo Over a year ago

@user17090811 you may need to rethink your update strategy. Appending to an array list is fine, but how would you delete from an array list? I think replacement is the best strategy to cover both new assignment, addition & deletion of elements.

colm.anseo Over a year ago

@user17090811 Q1: ok1 and ok2 are booleans - so they both need to be true for the logic to work (ok1==ok2 would not work as both may be false). Q2: Go maps are reference types, so no need to pass a map-pointer - as the function can update the maps in place. Q3: yes - if you want to iterate on the config via updates and don't care about keeping the old state - then yes there's no need to copy the map.

|

Collectives™ on Stack Overflow

Update Nested fields using Golang struct

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related