0

Given a sorted RDD[String], how can I save it to file with a header?

val header: String = "header row"
val foo: RDD[(Int, String)] = ???
foo.coalesce(1).sortByKey().saveAsTextFile("//some/path")

This should save the data in order (I believe). How can I inject the header in the top line?

Improve this question
3
  • What do you mean by "with a header"? column name? Commented Aug 10, 2016 at 2:11
  • No, just header info. It's not a CSV. The format is Metis input, which requires the first line to be a summary of the remaining data. Commented Aug 10, 2016 at 2:20
  • For a similar question sean-owen suggested creating RDD for the header line, and performing union between both RDDs (header and original-RDD) stackoverflow.com/questions/26157456/… Commented Aug 10, 2016 at 8:40
Related questions
13
Add a header before text file on save in Spark
74
How do I skip a header from CSV files in Spark?
0
Optimal solution for getting the header from CSV with Spark
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.