How to perform arithmetic operations on dataframe in spark using select and map?

Question

I am trying to manipulate dataframe using select in spark scala. I want to multiply column values by 10 using select but it fails. Also let me know how to perform map, flatmap, reducebykey operations on dataframe using select.

df1.select("Sales"*10,"Profit"*10).show()

df1.select("Sales").map(x=>x*10).show()

Help me to solve this issue! Thanks!

maxime G · Accepted Answer · 2021-02-22 00:52:32Z

1

you should use col instead of string :

df1
.select
 (
  (col("Sales")*10).as("Sales"),
  (col("Profit")*10).as("Profit")
)
.show()

answered Feb 22, 2021 at 0:52

maxime G

1,7981 gold badge17 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to perform arithmetic operations on dataframe in spark using select and map?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related