Write and read array data in CSV row

Question

I'm facing a problem and I can't solve it: I need to write a CSV file to process the CSV data. I'm doing it like this:

payload = {name: 'Dimitrius', errors: ['error1'] }
CSV.open(file_name, "a+") do |csv|
  csv.add_row(payload.values)
end

The CSV file was created and populated like a charm \o/ But, the array of errors in the CSV file was a 'string':

csv_text = File.read(file_name)
csv = CSV.parse(csv_text, headers: false)
puts csv # [["Dimitrius", "[\"error1\"]"]

The problem is, I must create so much 'workarounds' to transform the string "[\"error1\"]" into a valid array :(

My question is, this is the correct way to work with array values in CSV files?

Dimitrius Lachi · Accepted Answer · 2019-09-12 18:02:21Z

3

I found a way to get along with this problem :) Instead of initializing my payload to write in the CSV file with an array, I initialize it with a string with a separator:

payload = {name: 'Dimitrius', errors: ['error1', 'error2', 'error3'].join('|') }
CSV.open(file_name, "a+") do |csv|
  csv.add_row(payload.values)
end

Now, my CSV file will generate this result:

csv_text = File.read(file_name)
csv = CSV.parse(csv_text, headers: false)
puts csv # [["Dimitrius", "error1|error2|error3"]

And, I can easy transform this string "error1|error2|error3" into an array executing a .split like the follwing:

errors = "error1|error2|error3".split('|')
errors # ['error1', 'error2', 'error3']

I'm not sure if this is the best solution to work with arrays in CSV, but to my case works because the error string will never contain the operator |.

If someone has another idea, I will be glad to discuss about :)

edited Sep 12, 2019 at 18:02

answered Sep 12, 2019 at 14:16

Dimitrius Lachi

1,2872 gold badges11 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2020-06-20 09:12:55Z

I've stolen Dimitrius' idea of using a reserved character to denote an array and expanded on that by creating a custom data converter to convert strings containing the reserved character to arrays, the elements of which are then converted to appropriate data types.

Data

arr = [{ name: 'Dimitrius', errors: ['e1', 3, 4.1] },
       { name: 'Wilma',     errors: ['f1', 7, 9.5] }]

Write the CSV file

require 'csv'

FName = 't.csv'

I've assumed that if a substring between commas in the CSV file contains a specified reserved character it denotes an array, with the reserved character separating elements of the array.

RESERVED_CHAR = '|'

keys = arr.first.keys
  #=> [:name, :errors] 
rows = arr.map(&:values)
  #=> [["Dimitrius", ["e1", 3, 4.1]],
  #    ["Wilma",     ["f1", 7, 9.5]]]

CSV.open(FName, "wb") do |csv|
  csv << keys
  rows.each { |name,arr| csv << [name, arr.join(RESERVED_CHAR)] }
end
  #=> [["Dimitrius", ["e1", 3, 4.1]], ["Wilma", ["f1", 7, 9.5]]]

Let's see what was written.

File.read(FName).each_line { |line| p line }
"name,errors\n"
"Dimitrius,e1|3|4.1\n"
"Wilma,f1|7|9.5\n"

Read the CSV file

The CSV class has six built-in data converters (look under "Constants"). One may, however, add additional converters as needed. These convert values in the rows, but not the headers.

require 'csv'
FName = 't.csv'
RESERVED_CHAR = '|'

CSV::Converters[:array] = ->(v) do
  if v.include?(RESERVED_CHAR)
    v.split(RESERVED_CHAR).map do |e|
      case e
      when /\A\-?\d+\z/
        e.to_i
      when /\A\-?\d+\.\d+\z/
        e.to_f
      else
        e
      end
    end
  else
    v
  end
end

We may now read the CSV file into a CSV::Table object:

tbl = CSV.read(FName, headers: true, converters: :array)
  #=> #<CSV::Table mode:col_or_row row_count:3>

We can use this to extract the headers and rows:

headers, *body = tbl.to_a
  #=> [["name", "errors"],
  #    ["Dimitrius", ["e1", 3, 4.1]],
  #    ["Wilma", ["f1", 7, 9.5]]] 
headers
  #=> ["name", "errors"] 
body
  #=> [["Dimitrius", ["e1", 3, 4.1]],
  #    ["Wilma",     ["f1", 7, 9.5]]]

or individual columns:

tbl["name"]
  #=> ["Dimitrius", "Wilma"] 
tbl["errors"]
  #=> [["e1", 3, 4.1],
  #    ["f1", 7, 9.5]]

Note that the values in the arrays have been converted as desired, and the headers are now strings. If they are to be symbols that would require a separate step at the end.

Collectives™ on Stack Overflow

Write and read array data in CSV row

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related