11

I have an app where I allow users to import CSV data.

Everything was working until users started importing data with blank rows in the CSV file. I am using the following to grab the rows out of the CSV file: 

CSV.readlines(import_file, headers: true, skip_blanks: true)

I thought that if I added the option to skip_blanks that it would do that but it hasn't. Any ideas on how I can ignore the blank rows.

Thanks!

Improve this question
6
  • Are you sure the users are supplying empty rows, CSV files could be very funny at times Commented Dec 18, 2013 at 5:06
  • please try this form: CSV.new( ( open import_file ), headers: true, skip_blanks: true ).readlines Commented Dec 18, 2013 at 5:11
  • @majioa I just tried that but still has blank lines. I looped through the result and still have four empty rows that show up like this, ',,,' Commented Dec 18, 2013 at 5:21
  • You should avoid using .new on ruby IO objects except you are certain to close it, so it's better you use .open instead, readlines delegates to read which calls open and calls new view source here github.com/ruby/ruby/blob/… Commented Dec 18, 2013 at 5:24
  • @GrahamJackson those rows are not empty ',,,,' is four comma separated values Commented Dec 18, 2013 at 5:25

5 Answers 5

Reset to default
12

This should work

CSV.open(import_file, skip_blanks: true).reject { |row| row.all?(&:nil?) }

EDIT

You requested for readlines, it calls open in the CSV source code in the end but this is it:

CSV.readlines(import_file, skip_blanks: true).reject { |row| row.all?(&:nil?) }

I feel open would perform better though I have not done any bench marking

CSV.open(import_file, skip_blanks: true, headers: true).reject { |row| row.to_hash.values.all?(&:nil?) }

CSV.readlines(import_file, skip_blanks: true, headers: true).reject { |row| row.to_hash.values.all?(&:nil?) }

The above returns a collection of CSV::Row objects

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

This is a good answer but is there anyway to do the same thing but use readlines instead. (I would prefer to stick to that because the csv importer depends on the format that readlines provides the data in).
@GrahamJackson updated the answer to use readlines as you requested, if this works can you upvote and accept the answer?
Thank you for the help. This doesn't solve my problem but I appreciate it and will up vote it because it may help someone else. I need to remove the null values but still return a CSV object. This solution will return an array.
@GrahamJackson let me modify more, so you have your csv objects
Thank you. The new solution you provided does provide a collection of csv objects but doesn't end up removing the rows with nil values.
|
10

This returns a CSV::Table object.

CSV.parse(import_file, headers: true, skip_blanks: true).delete_if { |row| row.to_hash.values.all?(&:blank?) }

Improve this answer

1 Comment

the blank? was a nice solution for me, then works for '', ' ' and nil row data
4

The solution depends on if you read the CSV with or without headers, and if you want to keep working with a CSV::Table object or if you are happy with an array of CSV::Row

Without headers you'll get rows that looks something like:

[#<CSV::Row nil nil nil]

With headers:

[#<CSV::Row "name":nil "abbreviation":nil "age":nil]

SOLUTION

Without headers and return array of CSV::Row

table.reject { |row| row.all?(&:nil?) }

Without headers and return CSV::Table

table.delete_if { |row| row.to_hash.values.all?(&:nil?) }

With headers and return array of CSV::Row

table.reject { |row| row.to_hash.values.all?(&:nil?) }

Without headers and return CSV::Table

table.delete_if { |row| row.to_hash.values.all?(&:nil?) }

FURTHER READING

Ruby Documentation for delete_if

Improve this answer

Comments

1

Here is how I managed remove blank lines from csv:

table = CSV.parse(csv_text, headers: true)
table.delete_if { |row| row.to_hash.values.all?(&:blank?) }
Improve this answer

Comments

0

If you also want to filter out columns that include just whitespaces use:

row.all? { |column| column.nil? || column.strip.empty? }
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.