0

Playing around with golang parquet package and came across the function s3.NewS3FileWriter that accepts the following arguement []func(*s3manager.Uploader)

func NewS3FileWriter(
    ctx context.Context,
    bucket string,
    key string,
    uploaderOptions []func(*s3manager.Uploader),
    cfgs ...*aws.Config,
) (source.ParquetFile, error) {
    if activeS3Session == nil {
        sessLock.Lock()
        if activeS3Session == nil {
            activeS3Session = session.Must(session.NewSession())
        }
        sessLock.Unlock()
    }

    file := &S3File{
        ctx:             ctx,
        client:          s3.New(activeS3Session, cfgs...),
        writeDone:       make(chan error),
        uploaderOptions: uploaderOptions,
        BucketName:      bucket,
        Key:             key,
    }

    return file.Create(key)
}

What does []func(*s3manager.Uploader) mean? And how do I pass a modified version of s3Manager.Uploader? If I wanted to change the defaults for example.

2
  • 1
    Read it from left to right: Slice of function with argument pointer to s3manager.Uploader. Commented Jul 5, 2019 at 9:32
  • Hi, are you able to provide an example? Thanks Commented Jul 5, 2019 at 9:36

1 Answer 1

1

The uploaderOptions argument of NewS3FileWriter need to be filled with slice of functions during call. And each of the functions must have an argument with type of *s3manager.Uploader.

Example:

uploaderOptions := make([]func(*s3manager.Uploader), 0)

uploader1 := func (param *s3manager.Uploader) {
    fmt.Println("uploader 1", param)
}
uploaderOptions = append(uploaderOptions, uploader1)

uploader2 := func (param *s3manager.Uploader) {
    fmt.Println("uploader 2", param)
}
uploaderOptions = append(uploaderOptions, uploader2)

// ...

NewS3FileWriter(ctx, bucket, key, uploaderOptions)
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks - appreciate that

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.