Spark (Scala) Replace all values in string with new values

Question

I am new to spark, struck here by trying to replace values of string with new values.

Data in URL="a=1,
b=2,
c=3"
var header="a,b,c"
for (line <- Source.fromURL("/u/r/l").getLines) {
     if(header.contains(line.split("=")(0).toLowerCase().trim)) {
       header.replaceAll(line.split("=")(0).toLowerCase().trim,line.split("=")(1).toLowerCase().trim)
     }
      }

I am expecting my final output as

println(header)
1,2,3

How can I achieve this, when I run this, still getting "a,b,c"

Anahcolus · Accepted Answer · 2018-07-27 06:52:50Z

1

all you need is assignment as

for (line <- Source.fromURL("/u/r/l").getLines) {
     if(header.contains(line.split("=")(0).toLowerCase().trim)) {
       header = header.replaceAll(line.split("=")(0).toLowerCase().trim,line.split("=")(1).toLowerCase().trim)
     }
}

You can do it more functionally as

Source.fromURL("/u/r/l").getLines.map(_.split("=")).map(line => header = header.replaceAll(line(0).toLowerCase().trim,line(1).toLowerCase().trim))

going one step further you can change the url to Map and apply the mapping as

val kv = Source.fromURL("/u/r/l").getLines.map(_.split("=")).map(x => (x(0).toLowerCase.trim, x(1).toLowerCase.trim)).toMap
header = header.split(",").map(x => if(kv.keySet.contains(x)) kv(x);else x).mkString(",")

edited Jul 27, 2018 at 6:52

answered Jul 27, 2018 at 6:32

Anahcolus

42.1k6 gold badges75 silver badges101 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Spark (Scala) Replace all values in string with new values

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related