Type Mismatch while converting string to int in spark sql

Question

i am trying to add values of two column after converting from string to int using withcolumn in spark sql.

withColumn("totalDowntimetime",("zenerTime").cast(int)+("avalancheTime").cast(int)).show(truncate = false)

but it is throwing error as "

 Error:(36, 44) type mismatch;
     found   : String("zenerTime")
     required: org.apache.spark.sql.Column
        withColumn("totalDowntimetime",("zenerTime").cast(int)+("avalancheTime").cast(int)).show(truncate = false)

Very Much Appreciate your suggestion

notNull · Accepted Answer · 2020-08-19 19:44:47Z

1

Try refering your columns zenerTime,avalancheTime with col(<column_name>)

As required type is column

withColumn("totalDowntimetime",col("zenerTime").cast(int) + col("avalancheTime").cast(int)).show(truncate = false)

Example:

val df=Seq(("1","2")).toDF("zenerTime","avalancheTime")


import org.apache.spark.sql.functions._
import org.apache.spark.sql.types._

df.withColumn("totalDowntimetime",col("zenerTime").cast("int") + col("avalancheTime").cast("int")).show()
/*
+---------+-------------+-----------------+
|zenerTime|avalancheTime|totalDowntimetime|
+---------+-------------+-----------------+
|        1|            2|                3|
+---------+-------------+-----------------+
*/

edited Aug 19, 2020 at 19:44

answered Aug 19, 2020 at 19:37

notNull

31.8k4 gold badges41 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Aarti Agarwal Over a year ago

not sure why i am getting error as cannot resolve overload method cast.... any specific library , do i need to import.

notNull Over a year ago

I have added Example section in the answer try by importing functions and types.

Collectives™ on Stack Overflow

Type Mismatch while converting string to int in spark sql

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related