How to remove repeated columns using ruby FasterCSV

Question

I'm using Ruby 1.8 and FasterCSV.

The csv file I'm reading in has several repeated columns.

| acct_id | amount | acct_num | color | acct_id | acct_type | acct_num |
|     345 |  12.34 |      123 |   red |     345 | 'savings' |      123 |
|     678 |  11.34 |      432 | green |     678 | 'savings' |      432 |

...etc

I'd like to condense it to:

| acct_id | amount | acct_num | color | acct_type |
|     345 |  12.34 |      123 |   red | 'savings' |
|     678 |  11.34 |      432 | green | 'savings' |

Is there a general purpose way to do this?

Currently my solution is something like:

headers = CSV.read_line(file)
headers = CSV.read_line # get rid of garbage line between headers and data
FasterCSV.filter(file, :headers => headers) do |row|
  row.delete(6) #delete second acct_num field
  row.delete(4) #delete second acct_id field

  # additional processing on the data
  row['color'] = color_to_number(row['color'])
  row['acct_type'] = acct_type_to_number(row['acct_type'])
end

It works, but it's not elegant. E.g. I have a similar problem with another table with different indices. — mkirk
– mkirk, Commented Apr 5, 2011 at 20:11

steenslag · Accepted Answer · 2011-04-05 20:03:42Z

1

Assuming you want to get rid of the hardcoded deletions

  row.delete(6) #delete second acct_num field
  row.delete(4) #delete second acct_id field

Can be replaced by

row = row.to_hash

This will clobber duplicates. The rest of the posted code will keep working.

answered Apr 5, 2011 at 20:03

steenslag

80.2k16 gold badges144 silver badges174 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mkirk Over a year ago

but then the elements of the hash won't necessarily be in the same order as the elements of the row. This is a problem when I want to print out the results, as my fields won't match up with the header

steenslag Over a year ago

They will be in the same order in Ruby 1.9 . For 1.8.6 you'd need to specify the output manually, which I agree sort of defies the reason.

Collectives™ on Stack Overflow

How to remove repeated columns using ruby FasterCSV

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related