convert scala hashmap with List to java hashmap with java list

Question

I am new to scala and spark.I have below case class A

case class A(uniqueId : String, attributes: HashMap[String, List[String]])

Now I have a dataFrame of type A. I need to call a java function on each row of that DF. I need to convert Hashmap to Java HashMap and List to java list.. How can i do that.

I am trying to do following

val rddCaseClass = RDD[A]
val a = rddCaseClass.toDF().map ( x=> {
val rowData = x.getAs[java.util.HashMap[String,java.util.List[String]]]("attributes")
callJavaMethod(rowData)

But this is giving me error :

java.lang.ClassCastException: scala.collection.mutable.WrappedArray$ofRef cannot be cast to java.util.List

Please help.

Possible duplicate of Convertion of Scala typed collection to Java with an unbounded wildcard, etc. Collection<?> — talex
– talex, Commented Oct 9, 2016 at 10:50

Pawan B · Accepted Answer · 2016-10-09 11:06:23Z

1

You can convert Scala Wrapped array to Java List using scala.collection.JavaConversions

 val wrappedArray: WrappedArray[String] = WrappedArray.make(Array("Java", "Scala"))
 val javaList = JavaConversions.mutableSeqAsJavaList(wrappedArray)

JavaConversions.asJavaList can also be used but its deprecated: use mutableSeqAsJavaList instead

answered Oct 9, 2016 at 11:06

Pawan B

4,6332 gold badges27 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Kris · Accepted Answer · 2016-10-09 11:07:29Z

0

I think, you could use Seq instead of List for your parameters to work efficiently with List. This way it should work with most of the Seq implementations and no need to to convert the seqs like WrappedArray.

val rddCaseClass = RDD[A]
val a = rddCaseClass.toDF().map ( x=> {
val rowData = x.getAs[java.util.HashMap[String, Seq[String]]]("attributes")
callJavaMethod(rowData)

answered Oct 9, 2016 at 11:07

Kris

1,7641 gold badge14 silver badges15 bronze badges

Collectives™ on Stack Overflow

convert scala hashmap with List to java hashmap with java list

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related