According to DataFrames API, definition is:
public void foreach(scala.Function1<Row,scala.runtime.BoxedUnit> f)
Applies a function f to all rows.
But when I am trying like
Dataframe df = sql.read()
.format("com.databricks.spark.csv")
.option("header","true")
.load("file:///home/hadoop/Desktop/examples.csv");
df.foreach(x->
{
System.out.println(x);
});
I am getting compile time error. any mistake?
You can cast it as Java RDD in order to use the lambda as you which:
df.toJavaRDD().foreach(x->
System.out.println(x)
);
First extend scala.runtime.AbstractFunction1 and implement Serializable like below
public abstract class SerializableFunction1<T,R>
extends AbstractFunction1<T, R> implements Serializable
{
}
Now use this SerializableFunction1 class like below.
df.foreach(new SerializableFunction1<Row,BoxedUnit>(){
@Override
public BoxedUnit apply(Row row) {
System.out.println(row.get(0));
return BoxedUnit.UNIT;
}
});
Try with this code :
df.foreach(new VoidFunction<String>(){ public void call(String line) {
//your function code here
}});
If you want just to show df content, this much easier :
df.show();
foreachto print output. Third because there is a typo.DataFrameis with a big F ! I'm also voting to close the question for the matter.Function1<Row, BoxedUnit>does not seem to fit Java lambdas 2. if the parameter isdf.foreach(new AbstractFunction1<Row, BoxedUnit>() { @Override public BoxedUnit apply(Row arg0) { return null; } });, , it works just fine.