Golang POST request in JSON format from a csv file

Question

so I am trying to POST a csv file in JSON format to a website in Golang. I have been successful in POSTing a singular JSON file. However, that is not what I want the purpose of my program to be. Basically, I'm trying to create an account generator for a site. I want to be able to generate multiple accounts at once. I feel the best way to do this is with a csv file.

I've tried using encoding/csv to read the csv file then marshal to JSON. Also, ioutil.ReadFile(). However, the response from the site is, 'first name is mandatory field, last name is a mandatory field' etc etc. So, this obviously means the csv data is not going into JSON format. I'll show my code with the ioutil.ReadFile() below.

func main() {
file, _ := ioutil.ReadFile("accounts.csv")

    jsonConv, _ := json.Marshal(file)

    client := http.Client{}

    req, err := http.NewRequest("POST", "websiteUrlHere", bytes.NewBuffer(jsonConv))
    req.Header.Add("cookie", `"really long cookie goes here"`)
    req.Header.Set("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36")
    req.Header.Set("content-type", "application/json")

    resp, err := client.Do(req)
    if err != nil {
        fmt.Print(err)
    }
    defer resp.Body.Close()
}

(^ This is just a snippet of the code).

I'm still pretty new to all of this so please understand if the question lacks anything. I've also looked for similar questions but all I find is the use of a struct. I feel this wouldn't be applicable for this as the goal is to create unlimited accounts at once.

Hope the above is sufficient. Thank you.

ioutil.ReadFile just reads the raw data from the file. You need to actually parse it into a data-type, then output that data in json format. Google how to read a csv file into a struct, then you can use json.Marshal on that same struct. — super
– super, Commented Jun 20, 2020 at 20:54

GoForth · Accepted Answer · 2020-06-20 23:56:07Z

2

The issue with your code is actually that you're trying to convert a file to bytes with:

file, _ := ioutil.ReadFile("accounts.csv")

...and then you are AGAIN trying to convert that slice of bytes to JSON bytes with:

jsonConv, _ := json.Marshal(file)

Where the text contents of the file are stored as a slice of bytes in the variable file, and then that slice of bytes (the file contents in bytes form) is then being converted to a JSON array of bytes. So you are basically sending a JSON array of numbers...not good.

The normal flow here would be to take the file bytes and then create a Go struct(s) out of it. Once your Go objects are in place, THEN you would marshal to JSON. That converts the Go objects to a slice of bytes AFTER it has been converted to JSON text form.

So what you are missing is the Go structure middle step but you also should keep in mind that converting a Go struct to JSON bytes with json.Marshal() will only show fields that are exported. Also, usually you should use struct tags to customize exactly how the fields will show up.

Your best bet is just to stick with JSON, forget about the CSV. In your own code example, you are taking a CSV and then trying to convert it to JSON...so, then, just use JSON.

If you want to send multiple accounts, just make your Go structure a slice, which will marshal into a JSON array, which is basically what you are trying to do. The end result will be a JSON array of accounts. Here's a simple example:

package main

import (
    "fmt"
    "encoding/json"
)

type Account struct {
    Username string `json:"username"`
    Email string `json:"email"`
}

type AccountsRequest struct {
    Accounts []Account `json:"accounts"`
}

func main() {
    //gather account info
    acct1 := Account{Username: "tom", Email: "[email protected]"}
    acct2 := Account{Username: "dick", Email: "[email protected]"}
    acct3 := Account{Username: "harry", Email: "[email protected]"}
    
    //create request
    acctsReq := AccountsRequest{Accounts: []Account{acct1, acct2, acct3}}
    
    //convert to JSON bytes/data
    //jsonData, _ := json.Marshal(acctsReq)
    
    //debug/output
    jsonDataPretty, _ := json.MarshalIndent(acctsReq, "", "  ")
    fmt.Println(string(jsonDataPretty))
    
    //send request with data
    //...
}

Runnable here in playground.

The key is that the structs are set up and ready to go and the struct tags determine what the JSON field names will be (i.e. username & email for each account and accounts for the overall array of accounts).

Hope that helps. Drop a comment if you need more specific help.

answered Jun 20, 2020 at 23:56

GoForth

6768 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

chris Over a year ago

Thank you, this is very useful for personal use. However, I aim to distribute it to about 10 of my friends. They have no idea how to code so I feel a csv/json file would be much better. You think your method would still suffice for use without the code and an exe, etc?

GoForth Over a year ago

This is for your server side. So you see where I'm creating the 3 accounts at the start of main? Well, that's where you would parse their CSV and create one of those structs for each row. For the client side, you can just write an html file with a form input field that accepts the csv file upload. Then they upload CSV and you have a Go server listen for that request, parse the file and create those structs.

Michael Ernst · Accepted Answer · 2020-06-20 22:25:58Z

You need to parse the CSV file first and convert it into the list that you want:

package main


func main() {
   file, err := os.Open("file.csv")
   if err != nil {
      log.Fatal("failed opening file because: %s", err.Error())
   }
   r := csv.NewReader(file)

   records, err := r.ReadAll()
   if err != nil {
      log.Fatal(err)
   }
   fmt.Print(records)
}

The above code is parsing the list into a [][]string array. you will now need to iterate over that array and turn it into the json object that the page needs. Then you can send it. You can read more about the csv package here : https://golang.org/pkg/encoding/csv/

A word of advise: never ignore errors, they might give you useful information.

Collectives™ on Stack Overflow

Golang POST request in JSON format from a csv file

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related