java.lang.NumberFormatException: For input string: "" while creating Dataframe

Question

I am using cloudera vm. I have imported the products table from retail_db as a textfile with '|' as a fields separator (using sqoop).

Following is the table schema:

mysql> describe products;
product_id: int(11)
product_category_id: int(11)
product_name: varchar(45)
product_description: varchar(255)
product_price: float
product_image: varchar(255)

I want to create a Dataframe from this data.

I got no issue while using following code:

var products = sc.textFile("/user/cloudera/ex/products").map(r => {var p = r.split('|'); (p(0).toInt, p(1).toInt, p(2), p(3), p(4).toFloat, p(5))})
case class Products(productID: Int, productCategory: Int, productName: String, productDescription: String, productPrice: Float, productImage: String)
var productsDF = products.map(r => Products(r._1, r._2, r._3, r._4, r._5, r._6)).toDF()

productsDF.show()

But I got NumberFormatException exception for following code:

case class Products (product_id: Int, product_category_id: Int, product_name: String, product_description: String, product_price: Float, product_image: String)
val productsDF = sc.textFile("/user/cloudera/ex/products").map(_.split("|")).map(p => Products(p(0).trim.toInt, p(1).trim.toInt, p(2), p(3), p(4).trim.toFloat, p(5))).toDF()
productsDF.show()

java.lang.NumberFormatException: For input string: ""

Why is that I am getting exception in 2nd code even though it is same as that of 1st one?

1009|45|Diamond Fear No Evil Compound Bow Package||599.99|images.acmesports.sports/… 1010|46|DBX Vector Series Men's Nylon Life Vest||19.98|images.acmesports.sports/… — ANP
– ANP, Commented Jul 26, 2017 at 5:30

koiralo · Accepted Answer · 2017-07-26 05:53:28Z

3

The error is due to _.split("|") in second part of your code

You need to use _.split('|') or _.split("\\|") or _.split("""\|""") or Pattern.quote("|")

If you use "|" it tries to split with regular expression and | is or in the regular expression, so it does not matches anything and returns empty string ""

Hope this helps!

edited Jul 26, 2017 at 5:53

answered Jul 26, 2017 at 5:42

koiralo

23.2k6 gold badges57 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

java.lang.NumberFormatException: For input string: "" while creating Dataframe

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related