Pass List[String] to function that takes f(args: String*) scala

Question

I need to read in specific parquet files with spark, I know this can be done like so:

sqlContext
    .read
    .parquet("s3://bucket/key", "s3://bucket/key")

Right now I have a List[String] object with all these s3 paths in it but I don't know how I can pass this programmatically to the parquet function in Scala? There are way to many files to do it manually, any ideas how to get the files into the parquet function programmatically?

You are looking for splat operator: .parquet(listOfStrings:_*) — Dima
– Dima, Commented Jul 8, 2016 at 2:33

Community · Accepted Answer · 2017-05-23 12:22:29Z

3

I've answer a similar question earlier concerning repeated parameters here.

As @Dima mentioned, you are looking for a splat operator because .parquet expected repeated arguments :

sqlContext.read.parquet(listOfStrings:_*)

More on repeated arguments in the Scala Language Specification seciton 4.6.2

Although it's the specs of scala 2.9, this part didn't change.

edited May 23, 2017 at 12:22

CommunityBot

11 silver badge

answered Jul 8, 2016 at 8:39

eliasah

40.5k12 gold badges128 silver badges159 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pass List[String] to function that takes f(args: String*) scala

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related