1

My question is almost same as Add a header before text file on save in Spark. The difference is that my header RDD is 

String headerSTR = "inc_id,po_id,ass,inci_type,cat,sub_cat";
JavaRDD<String> PMheader = jsc.parallelize(Arrays.asList(headerSTR));

And my lines RDD is of PM Table type.

JavaRDD<PMTable>rdd_records=noheader.map(new Function<String,PMTable>(){---
PMTable sd = new PMTable(----);
return sd;});
rdd_records.saveAsTextFile();
mergeAllFiles();

I have merged all the result files to a single csv file which does not contain header .Now I need to get union of header rdd and lines rdd .But the method union(JavaRDD) in the type JavaRDD is not applicable for the arguments (JavaRDD of PMTable type). So how can i get the union of header and lines using spark-java api.

Thanks in advance.

Improve this question
1
  • Now I am able to get the union of both headers and lines.I have created a headerRDD of < PMTable> type. And take the union of both. Commented Dec 30, 2016 at 4:07
Related questions
74
How do I skip a header from CSV files in Spark?
8
Writing a RDD to a csv
0
skip header of csv while reading multiple files into rdd in scala
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.