Golang BigQuery: Insert array into table

Question

I'm trying to insert an array into a table using the bigquery golang library.

I have a table schema that looks like this. The array is the quality column, has the Repeated.

    tableSchema := bigquery.Schema{
        {Name: "issue_id", Type: bigquery.StringFieldType},
        {Name: "quality", Type: bigquery.StringFieldType, Repeated: true},
        {Name: "creation_date_ts", Type: bigquery.TimestampFieldType},
    }

This is how I defined the record:

type escalationsRecord struct {
    IssueID      bigquery.NullString    `bigquery:"issue_id"`
    Quality      []string               `bigquery:"quality"`
    CreationTime bigquery.NullTimestamp `bigquery:"creation_date_ts"`
}

This is how I create a record:

    record := escalationsRecord{
            IssueID:      bigquery.NullString{StringVal: fmt.Sprint(int(issue.Number)), Valid: true},
            Quality:      qualityArray,
            CreationTime: bigquery.NullTimestamp{Timestamp: issue.CreatedAt.Time, Valid: true},
        }
records = append(records, &record)

This is how I put the records to BigQuery

inserter := table.Inserter()
err2 := inserter.Put(b.ctx, records)

The quality column is NULL when I look at it in bigqyery. There are no errors. The array contains elements. Any idea how to properly insert arrays? I can't find anything in the docs.

Can you show how the qualityArray is built? I'm testing with something similar to what you have and don't have any issues storing multiple values. I wonder if the issue is defining the schema value as bigquery.StringFieldType and that doesn't match what the struct has (assuming it's []string). You may try using the bigquery.InferSchema() method of defining the schema from a struct to ensure it matches. — Brian Wagner
– Brian Wagner, Commented Dec 6, 2022 at 0:03
@BrianWagner using InferSchema did the trick! Mind adding this as an answer? — Dan Dinu
– Dan Dinu, Commented Dec 22, 2022 at 15:00

Dan Dinu · Accepted Answer · 2022-12-22 15:02:35Z

2

What @BrianWagner suggested in his comment works. The only change I did was to use the InferSchema() instead of defining it myself.

So instead of

  tableSchema := bigquery.Schema{
        {Name: "issue_id", Type: bigquery.StringFieldType},
        {Name: "quality", Type: bigquery.StringFieldType, Repeated: true},
        {Name: "creation_date_ts", Type: bigquery.TimestampFieldType},
    }

I now have

    tableSchema, err := bigquery.InferSchema(escalationsRecord{})
    if err != nil {
        log.Fatal(err)
    }

answered Dec 22, 2022 at 15:02

Dan Dinu

33.5k25 gold badges85 silver badges120 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Golang BigQuery: Insert array into table

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related