Spark CSV Dataframe Load ,File not found

Question

How to catch exception , if my file is not found in given location in spark csv read?

try {
    val df = sqlCxt.read.format("csv")
    .option("delimiter", "|")
    .load("C:/Users/swapnil.shirke/Downloads/dataset/u.item");

}
catch {

case ex: FileNotFoundException => {
println(" file missing exception")
}
case ex: Exception => ex.printStackTrace()
}

given try catch condition not worked.

have you checked in your executor logs whether that message is printed or not? — vatsal mevada
– vatsal mevada, Commented Sep 27, 2017 at 7:44

Anahcolus · Accepted Answer · 2017-09-27 08:02:54Z

4

In scala, if the file path is not correct then the exception is org.apache.spark.sql.AnalysisException: Path does not exist: .... so the correct way is

try {
  val df = sqlCxt.read.format("csv")
    .option("delimiter", "|")
    .load("/Users/swapnil.shirke/Downloads/dataset/u.item");

}
catch {
  case ex: AnalysisException => {
    println(" file missing exception")
  }
  case ex: Exception => ex.printStackTrace()
}

I hope the answer is helpful

answered Sep 27, 2017 at 8:02

Anahcolus

42.1k6 gold badges75 silver badges101 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Spark CSV Dataframe Load ,File not found

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related