golang json array unmarshal into type struct

Question

What is causing this to not break apart json? Anyone have an idea on how type is wrong?

With error of:

{"Rates":[{"Symbol":"EURAUD","Bid":"1.45492","Ask":"1.45608","Spread":"11.60","ProductType":"1",},{"Symbol":"Copper","Bid":"2.6068","Ask":"2.6088","Spread":"2.00","ProductType":"3",},{"Symbol":"AUDNZD","Bid":"1.08999","Ask":"1.09177","Spread":"17.80","ProductType":"1",},{"Symbol":"EURSEK","Bid":"9.63786","Ask":"9.65569","Spread":"178.30","ProductType":"1",},{"Symbol":"CADJPY","Bid":"81.629","Ask":"81.708","Spread":"7.90","ProductType":"1",},{"Symbol":"USDCHF","Bid":"0.99463","Ask":"0.99527","Spread":"6.40","ProductType":"1",},{"Symbol":"USDCNH","Bid":"6.8973","Ask":"6.8993","Spread":"20.00","ProductType":"1",},{"Symbol":"US30","Bid":"20950.00","Ask":"20952.00","Spread":"2.00","ProductType":"2",},{"Symbol":"XAGUSD","Bid":"17.202","Ask":"17.25","Spread":"4.80","ProductType":"5",},{"Symbol":"USDSEK","Bid":"8.84794","Ask":"8.85542","Spread":"74.80","ProductType":"1",},{"Symbol":"AUDCHF","Bid":"0.74417","Ask":"0.74588","Spread":"17.10","ProductType":"1",},{"Symbol":"GER30","Bid":"12431.05","Ask":"12433.45","Spread":"2.40","ProductType":"2",},{"Symbol":"USOil","Bid":"49.16","Ask":"49.21","Spread":"5.00","ProductType":"3",},{"Symbol":"GBPNZD","Bid":"1.88546","Ask":"1.88762","Spread":"21.60","ProductType":"1",},{"Symbol":"EURCAD","Bid":"1.48748","Ask":"1.48893","Spread":"14.50","ProductType":"1",},{"Symbol":"EURUSD","Bid":"1.08977","Ask":"1.08997","Spread":"2.00","ProductType":"1",},{"Symbol":"AUS200","Bid":"5922.00","Ask":"5924.00","Spread":"2.00","ProductType":"2",},{"Symbol":"EURJPY","Bid":"121.512","Ask":"121.57","Spread":"5.80","ProductType":"1",},{"Symbol":"EURGBP","Bid":"0.84132","Ask":"0.84208","Spread":"7.60","ProductType":"1",},{"Symbol":"EURNOK","Bid":"9.34136","Ask":"9.36364","Spread":"222.80","ProductType":"1",},{"Symbol":"USDCAD","Bid":"1.36524","Ask":"1.36588","Spread":"6.40","ProductType":"1",},{"Symbol":"GBPCHF","Bid":"1.28753","Ask":"1.28922","Spread":"16.90","ProductType":"1",},{"Symbol":"GBPAUD","Bid":"1.72838","Ask":"1.7303","Spread":"19.20","ProductType":"1",},{"Symbol":"USDJPY","Bid":"111.51","Ask":"111.537","Spread":"2.70","ProductType":"1",},{"Symbol":"USDNOK","Bid":"8.57607","Ask":"8.58684","Spread":"107.70","ProductType":"1",},{"Symbol":"AUDCAD","Bid":"1.02173","Ask":"1.02347","Spread":"17.40","ProductType":"1",},{"Symbol":"FRA40","Bid":"5259.60","Ask":"5267.30","Spread":"7.70","ProductType":"2",},{"Symbol":"AUDUSD","Bid":"0.74858","Ask":"0.74899","Spread":"4.10","ProductType":"1",},{"Symbol":"USDHKD","Bid":"7.77769","Ask":"7.77956","Spread":"18.70","ProductType":"1",},{"Symbol":"NZDCHF","Bid":"0.68192","Ask":"0.68406","Spread":"21.40","ProductType":"1",},{"Symbol":"EURTRY","Bid":"3.86851","Ask":"3.87478","Spread":"62.70","ProductType":"1",},{"Symbol":"AUDJPY","Bid":"83.469","Ask":"83.543","Spread":"7.40","ProductType":"1",},{"Symbol":"USDZAR","Bid":"13.3464","Ask":"13.3941","Spread":"477.00","ProductType":"1",},{"Symbol":"Bund","Bid":"161.78","Ask":"161.81","Spread":"3.00","ProductType":"4",},{"Symbol":"USDMXN","Bid":"18.81249","Ask":"18.83178","Spread":"192.90","ProductType":"1",},{"Symbol":"USDTRY","Bid":"3.54925","Ask":"3.5536","Spread":"43.50","ProductType":"1",},{"Symbol":"USDOLLAR","Bid":"12232.00","Ask":"12237.00","Spread":"5.00","ProductType":"7",},{"Symbol":"JPN225","Bid":"19195.00","Ask":"19205.00","Spread":"10.00","ProductType":"2",},{"Symbol":"UK100","Bid":"7197.80","Ask":"7198.90","Spread":"1.10","ProductType":"2",},{"Symbol":"HKG33","Bid":"24650.00","Ask":"24655.00","Spread":"5.00","ProductType":"2",},{"Symbol":"CADCHF","Bid":"0.72748","Ask":"0.72979","Spread":"23.10","ProductType":"1",},{"Symbol":"NAS100","Bid":"5582.80","Ask":"5583.80","Spread":"1.00","ProductType":"2",},{"Symbol":"NGAS","Bid":"3.2645","Ask":"3.2755","Spread":"11.00","ProductType":"3",},{"Symbol":"ZARJPY","Bid":"8.323","Ask":"8.361","Spread":"3.80","ProductType":"1",},{"Symbol":"GBPCAD","Bid":"1.76724","Ask":"1.76912","Spread":"18.80","ProductType":"1",},{"Symbol":"ESP35","Bid":"10712.00","Ask":"10720.00","Spread":"8.00","ProductType":"2",},{"Symbol":"GBPUSD","Bid":"1.29452","Ask":"1.29527","Spread":"7.50","ProductType":"1",},{"Symbol":"SPX500","Bid":"2384.18","Ask":"2384.68","Spread":"5.00","ProductType":"2",},{"Symbol":"GBPJPY","Bid":"144.336","Ask":"144.448","Spread":"11.20","ProductType":"1",},{"Symbol":"EUSTX50","Bid":"3554.00","Ask":"3555.00","Spread":"1.00","ProductType":"2",},{"Symbol":"TRYJPY","Bid":"31.378","Ask":"31.44","Spread":"6.20","ProductType":"1",},{"Symbol":"NZDCAD","Bid":"0.93642","Ask":"0.93862","Spread":"22.00","ProductType":"1",},{"Symbol":"EURNZD","Bid":"1.58644","Ask":"1.58916","Spread":"27.20","ProductType":"1",},{"Symbol":"XAUUSD","Bid":"1267.79","Ask":"1268.26","Spread":"47.00","ProductType":"5",},{"Symbol":"NZDUSD","Bid":"0.68587","Ask":"0.68692","Spread":"10.50","ProductType":"1",},{"Symbol":"NZDJPY","Bid":"76.489","Ask":"76.607","Spread":"11.80","ProductType":"1",},{"Symbol":"UKOil","Bid":"51.84","Ask":"51.89","Spread":"5.00","ProductType":"3",},{"Symbol":"CHFJPY","Bid":"112.02","Ask":"112.148","Spread":"12.80","ProductType":"1",},{"Symbol":"EURCHF","Bid":"1.08416","Ask":"1.08459","Spread":"4.30","ProductType":"1",}]} {"Rates":[{"Symbol":"EURAUD","Bid":"1.45492","Ask":"1.45608","Spread":"11.60","ProductType":"1"},{"Symbol":"Copper","Bid":"2.6068","Ask":"2.6088","Spread":"2.00","ProductType":"3"},{"Symbol":"AUDNZD","Bid":"1.08999","Ask":"1.09177","Spread":"17.80","ProductType":"1"},{"Symbol":"EURSEK","Bid":"9.63786","Ask":"9.65569","Spread":"178.30","ProductType":"1"},{"Symbol":"CADJPY","Bid":"81.629","Ask":"81.708","Spread":"7.90","ProductType":"1"},{"Symbol":"USDCHF","Bid":"0.99463","Ask":"0.99527","Spread":"6.40","ProductType":"1"},{"Symbol":"USDCNH","Bid":"6.8973","Ask":"6.8993","Spread":"20.00","ProductType":"1"},{"Symbol":"US30","Bid":"20950.00","Ask":"20952.00","Spread":"2.00","ProductType":"2"},{"Symbol":"XAGUSD","Bid":"17.202","Ask":"17.25","Spread":"4.80","ProductType":"5"},{"Symbol":"USDSEK","Bid":"8.84794","Ask":"8.85542","Spread":"74.80","ProductType":"1"},{"Symbol":"AUDCHF","Bid":"0.74417","Ask":"0.74588","Spread":"17.10","ProductType":"1"},{"Symbol":"GER30","Bid":"12431.05","Ask":"12433.45","Spread":"2.40","ProductType":"2"},{"Symbol":"USOil","Bid":"49.16","Ask":"49.21","Spread":"5.00","ProductType":"3"},{"Symbol":"GBPNZD","Bid":"1.88546","Ask":"1.88762","Spread":"21.60","ProductType":"1"},{"Symbol":"EURCAD","Bid":"1.48748","Ask":"1.48893","Spread":"14.50","ProductType":"1"},{"Symbol":"EURUSD","Bid":"1.08977","Ask":"1.08997","Spread":"2.00","ProductType":"1"},{"Symbol":"AUS200","Bid":"5922.00","Ask":"5924.00","Spread":"2.00","ProductType":"2"},{"Symbol":"EURJPY","Bid":"121.512","Ask":"121.57","Spread":"5.80","ProductType":"1"},{"Symbol":"EURGBP","Bid":"0.84132","Ask":"0.84208","Spread":"7.60","ProductType":"1"},{"Symbol":"EURNOK","Bid":"9.34136","Ask":"9.36364","Spread":"222.80","ProductType":"1"},{"Symbol":"USDCAD","Bid":"1.36524","Ask":"1.36588","Spread":"6.40","ProductType":"1"},{"Symbol":"GBPCHF","Bid":"1.28753","Ask":"1.28922","Spread":"16.90","ProductType":"1"},{"Symbol":"GBPAUD","Bid":"1.72838","Ask":"1.7303","Spread":"19.20","ProductType":"1"},{"Symbol":"USDJPY","Bid":"111.51","Ask":"111.537","Spread":"2.70","ProductType":"1"},{"Symbol":"USDNOK","Bid":"8.57607","Ask":"8.58684","Spread":"107.70","ProductType":"1"},{"Symbol":"AUDCAD","Bid":"1.02173","Ask":"1.02347","Spread":"17.40","ProductType":"1"},{"Symbol":"FRA40","Bid":"5259.60","Ask":"5267.30","Spread":"7.70","ProductType":"2"},{"Symbol":"AUDUSD","Bid":"0.74858","Ask":"0.74899","Spread":"4.10","ProductType":"1"},{"Symbol":"USDHKD","Bid":"7.77769","Ask":"7.77956","Spread":"18.70","ProductType":"1"},{"Symbol":"NZDCHF","Bid":"0.68192","Ask":"0.68406","Spread":"21.40","ProductType":"1"},{"Symbol":"EURTRY","Bid":"3.86851","Ask":"3.87478","Spread":"62.70","ProductType":"1"},{"Symbol":"AUDJPY","Bid":"83.469","Ask":"83.543","Spread":"7.40","ProductType":"1"},{"Symbol":"USDZAR","Bid":"13.3464","Ask":"13.3941","Spread":"477.00","ProductType":"1"},{"Symbol":"Bund","Bid":"161.78","Ask":"161.81","Spread":"3.00","ProductType":"4"},{"Symbol":"USDMXN","Bid":"18.81249","Ask":"18.83178","Spread":"192.90","ProductType":"1"},{"Symbol":"USDTRY","Bid":"3.54925","Ask":"3.5536","Spread":"43.50","ProductType":"1"},{"Symbol":"USDOLLAR","Bid":"12232.00","Ask":"12237.00","Spread":"5.00","ProductType":"7"},{"Symbol":"JPN225","Bid":"19195.00","Ask":"19205.00","Spread":"10.00","ProductType":"2"},{"Symbol":"UK100","Bid":"7197.80","Ask":"7198.90","Spread":"1.10","ProductType":"2"},{"Symbol":"HKG33","Bid":"24650.00","Ask":"24655.00","Spread":"5.00","ProductType":"2"},{"Symbol":"CADCHF","Bid":"0.72748","Ask":"0.72979","Spread":"23.10","ProductType":"1"},{"Symbol":"NAS100","Bid":"5582.80","Ask":"5583.80","Spread":"1.00","ProductType":"2"},{"Symbol":"NGAS","Bid":"3.2645","Ask":"3.2755","Spread":"11.00","ProductType":"3"},{"Symbol":"ZARJPY","Bid":"8.323","Ask":"8.361","Spread":"3.80","ProductType":"1"},{"Symbol":"GBPCAD","Bid":"1.76724","Ask":"1.76912","Spread":"18.80","ProductType":"1"},{"Symbol":"ESP35","Bid":"10712.00","Ask":"10720.00","Spread":"8.00","ProductType":"2"},{"Symbol":"GBPUSD","Bid":"1.29452","Ask":"1.29527","Spread":"7.50","ProductType":"1"},{"Symbol":"SPX500","Bid":"2384.18","Ask":"2384.68","Spread":"5.00","ProductType":"2"},{"Symbol":"GBPJPY","Bid":"144.336","Ask":"144.448","Spread":"11.20","ProductType":"1"},{"Symbol":"EUSTX50","Bid":"3554.00","Ask":"3555.00","Spread":"1.00","ProductType":"2"},{"Symbol":"TRYJPY","Bid":"31.378","Ask":"31.44","Spread":"6.20","ProductType":"1"},{"Symbol":"NZDCAD","Bid":"0.93642","Ask":"0.93862","Spread":"22.00","ProductType":"1"},{"Symbol":"EURNZD","Bid":"1.58644","Ask":"1.58916","Spread":"27.20","ProductType":"1"},{"Symbol":"XAUUSD","Bid":"1267.79","Ask":"1268.26","Spread":"47.00","ProductType":"5"},{"Symbol":"NZDUSD","Bid":"0.68587","Ask":"0.68692","Spread":"10.50","ProductType":"1"},{"Symbol":"NZDJPY","Bid":"76.489","Ask":"76.607","Spread":"11.80","ProductType":"1"},{"Symbol":"UKOil","Bid":"51.84","Ask":"51.89","Spread":"5.00","ProductType":"3"},{"Symbol":"CHFJPY","Bid":"112.02","Ask":"112.148","Spread":"12.80","ProductType":"1"},{"Symbol":"EURCHF","Bid":"1.08416","Ask":"1.08459","Spread":"4.30","ProductType":"1"}]}

panic: json: cannot unmarshal string into Go value of type main.MsgRatesArray

goroutine 1 [running]:

main.main()
    /tmp/test.go:50 +0x52c

With this code:

package main

import (
        "log"
        "fmt"
        "net/http"
        "bytes"
        "io/ioutil"
        "strings"
        "github.com/pquerna/ffjson/ffjson"
)


type MsgRatesArray struct {
        RateQuote []MsgRateQuoteJson `json:"Rates"`
}
type MsgRateQuoteJson struct {
        SymbolName string `json:"Symbol"`
        SymbolBid int64 `json:"Bid"`
        SymbolAsk int64 `json:"Ask"`
        SymbolSpread int64 `json:"Spread"`
        SymbolPT string `json:"ProductType"`
}

var respBytes []byte

func main() {
    var msg MsgRatesArray
    response,err := http.Get("https://ratesjson.fxcm.com/DataDisplayer")
    if err != nil {
        log.Fatal(err)
    }
    defer response.Body.Close()
    respBytes, err := ioutil.ReadAll(response.Body)
    //Get bad JSON into string
    jsonBytes := respBytes[bytes.Index(respBytes, []byte("{")):bytes.LastIndex(respBytes, []byte("}"))+1]
    jsonString := string(jsonBytes)
    fmt.Println(jsonString)
    // Use a positive number to indicate max replacement count to fix bad JSON string so we can remove comma in JSON ARRAY.
    result := strings.Replace(jsonString, "\",}", "\"}", -1)
    fmt.Println(result)
    // Turn GOOD JSON string back to JSON BYTES (BIN)
    jsonBytes2, err := ffjson.Marshal(result)
    if err != nil {
        panic(err)
    }
    // Parse JSON !
    err = ffjson.Unmarshal(jsonBytes2, &msg)
    if err != nil {
        panic(err)
    }
}

What is wrong with type for json array?

mkopriva · Accepted Answer · 2017-04-29 21:36:18Z

2

In the returned json Bid, Ask, and Spread are json strings not integers, so change your type definition to this:

type MsgRateQuoteJson struct {
    SymbolName   string `json:"Symbol"`
    SymbolBid    string `json:"Bid"`
    SymbolAsk    string `json:"Ask"`
    SymbolSpread string `json:"Spread"`
    SymbolPT     string `json:"ProductType"`
}

And marshaling a json string to get json bytes is not the correct way, just convert the string to a byte slice like this:

jsonBytes2 := []byte(result)

... and you're good to go:

// Parse JSON !
err = ffjson.Unmarshal(jsonBytes2, &msg)
if err != nil {
    panic(err)
}

Edit:

If you want to convert those strings into specific types during the json unmarshaling, you can do so by defining an UnmarshalJSON method on the *MsgRateQuoteJson type, plus with the help of the strconv package, like this:

type MsgRateQuoteJson struct {
    SymbolName   string  `json:"Symbol"`
    SymbolBid    float64 `json:"Bid"`
    SymbolAsk    float64 `json:"Ask"`
    SymbolSpread float64 `json:"Spread"`
    SymbolPT     int64   `json:"ProductType"`
}

func (msg *MsgRateQuoteJson) UnmarshalJSON(data []byte) (err error) {
    m := map[string]string{}
    if err = ffjson.Unmarshal(data, &m); err != nil {
        return err
    }

    msg.SymbolName = m["Symbol"]
    if msg.SymbolBid, err = strconv.ParseFloat(m["Bid"], 64); err != nil {
        return err
    }
    if msg.SymbolAsk, err = strconv.ParseFloat(m["Ask"], 64); err != nil {
        return err
    }
    if msg.SymbolSpread, err = strconv.ParseFloat(m["Spread"], 64); err != nil {
        return err
    }
    if msg.SymbolPT, err = strconv.ParseInt(m["ProductType"], 10, 64); err != nil {
        return err
    }
    return nil
}

edited Apr 29, 2017 at 21:36

answered Apr 29, 2017 at 20:49

mkopriva

38.7k4 gold badges73 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

user2777145 Over a year ago

How could this be converted to floats and int not strings?

mkopriva Over a year ago

Are you sure you want ints and not floats? All those strings contain a floating point number...

user2777145 Over a year ago

Some are floats ( bid/ask/spread) and one is int (producttype) rest is string ( symbol )

user2777145 Over a year ago

Only problem with that func is it drops off symbol...only dose number conversion.

mkopriva Over a year ago

Good catch @user2777145, fixed it.

|

Collectives™ on Stack Overflow

golang json array unmarshal into type struct

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related