Functional Style reading large csv file in Scala

Question

I am new to functional-style programming and scala, so my question might seem to be bit primitive.

Is there a specific way to read csv file in scala using functional style? Also, how the inner joins are performed for combining 2 csv files in scala using functional style?

I know spark and generally use data frame but don't have any idea in scala and finding it tough to search on google as well, since don't have much knowledge about it. Also, if anyone knows good links for the functional style programming for scala it would be great help.

Read Paul Chiusano, Rúnar Bjarnason "Functional Programming in Scala" — Emiliano Martinez
– Emiliano Martinez, Commented Oct 28, 2020 at 20:19
Some examples from some smart guys - stackoverflow.com/questions/7390322/… — jacks
– jacks, Commented Oct 28, 2020 at 23:54
Does this answer your question? Scala: Iterate over CSV files in a functional way? — tgdavies
– tgdavies, Commented Oct 31, 2020 at 12:05

simpadjo · Accepted Answer · 2020-10-29 14:45:15Z

1

The question is indeed too broad.

Is there a specific way to read csv file in scala using functional style?

So far I don't know of a king's road to parse CSVs completely hassle-free. CSV parsing includes

going through the input line-by-line
understanding, what to do with the (optional) header
accurately parsing each line each line according to the CSV specification
turning line parts into a business object

I recommend to

turn your input into Iterator[String]
split each line into parts using a library of your choice (e.g. opencsv)
manually create a desired domain object from the line parts

Here is an simple example which (ignores error handling and potential header)

case class Person(name: String, street: String)
val lineParser = new CSVParserBuilder().withSeparator(',').build()
val lines: Iterator[String] = Source.fromInputStream(new FileInputStream("file.csv")).getLines()
val parsedObjects: Iterator[Person] = lines.map(line => {
  val parts: Array[String] = lineParser.parseLine(line)
  Person(parts(0), parts(1))
})

edited Oct 29, 2020 at 14:45

answered Oct 28, 2020 at 18:48

simpadjo

4,0171 gold badge20 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ADK Over a year ago

Thanks, sorry for the broad question. But how do functional style works in case of reading CSV file in scala? Is there any example for it to refer?

simpadjo Over a year ago

Ok, I added an example

Collectives™ on Stack Overflow

Functional Style reading large csv file in Scala

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related