Updating a string variable from within nested for loop in Scala

Question

I have a Dataframe which I am trying converting to html table syntax as a string in Scala:

sample_df: which is of type DataFrame = Dataset[Row]

|Name|Country|Occupation|Years_Experience|
|Bob |USA    | Engineer |  5             |
|John|CANADA | Sales    |  3             |

I already have the html headers converted however need to have the html rows and values in a string variable as below html_string:

<tr>
      <td>Bob</td>
      <td>USA</td>
      <td>Engineer</td>
      <td>5</td>
</tr>
<tr>
      <td>John</td>
      <td>CANADA</td>
      <td>Sales</td>
      <td>3</td>
</tr>

However when I try nested loop method below, the html_string variable is not being updated to what I am expecting above. It results in a blank. What am I doing wrong?

Code Attempt:

  var html_string = """"""

  for (i <- sample_df) {
    html_string = html_string + "<tr>"
    for (g <- i.toSeq)  {
      html_string = html_string + "<td>" + g + "</td>"
    }
    html_string = html_string + "</tr>"
  }
  println(html_string)

Thanks!

seems like spark is a wrong tool for this. Just do it as a vanilla scala program — Dima
– Dima, Commented May 7, 2022 at 15:03

Dima · Accepted Answer · 2022-05-07 14:34:13Z

2

Not sure what you mean by "dataframe" in this context (you can't iterate spark dataframes like this), but assuming you are talking about a normal sequence, this works for me.

Having said that, this is a very bad way of doing this in scala. The basic rule of thumb when writing scala code is never use mutable variables unless. you know exactly why you must in a specific case (which is extremely rare).

   val html_string = df
     .map { _.mkString("<td>", "</td><td>", "</td>" }
     .mkString("<tr>", "</tr><tr>",  "</tr>")

answered May 7, 2022 at 14:34

Dima

40.6k6 gold badges54 silver badges80 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Techno04335 Over a year ago

the dataframe type I have is "type DataFrame = Dataset[Row]"

Techno04335 Over a year ago

to your point, I have converted the DataType to Sequence via "val html_string_new = html_string.collect().toSeq". and utilized the mkString option you have proposed. Thanks!

Collectives™ on Stack Overflow

Updating a string variable from within nested for loop in Scala

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related