Mapping CSV columns

Question

I have the following code where I am trying to map columns from the input file to output.

I have written it using multiple loops. Is there a way to write this more efficiently?

input.csv:

  Name, Age, Gender, Nation
  Joe, 18, Male, British

output.csv:

First_name, Age_Years, Gender_F_M, Religion, Nationality
Joe, 18, Male, , British

code:

import csv

renamed_headers = {
    "First_name": "Name",
    "Age_Years":"Age",
    "Gender_F_M":"Gender",
    "Religion": None,
    "Nationality": "Nation",
}

with open("input.csv") as input_file, open(r"output.csv", "w", newline="") as output_file:
    reader = csv.DictReader(input_file, delimiter=",")
    writer = csv.writer(output_file, delimiter=",")

    # write headers
    header_line = []
    for header_name in renamed_headers.keys():
        header_line.append(header_name)
    writer.writerow(header_line)

    # write values
    for item in reader:
        row_to_write = []
        print(item)
        for value in renamed_headers.values():
            if value:
                row_to_write.append(item[value])
            else:
                row_to_write.append("")

        writer.writerow(row_to_write)

I suspect that all the parts involving "writerow" should be one level of indentation deeper (so that we are still in the "with" block). — SylvainD
– SylvainD, Commented Jun 1, 2021 at 11:06
The current question title, which states your concerns about the code, applies to too many questions on this site to be useful. The site standard is for the title to simply state the task accomplished by the code. Please see How to Ask for examples, and revise the title accordingly. — Mast
– Mast ♦, Commented Jun 1, 2021 at 11:07
ah apologies the renamed header names were the wrong way round — pythonicwiz
– pythonicwiz, Commented Jun 1, 2021 at 11:07
@Mast and if theres a more efficient way to write it - i feel as if I am looping too much in write row - what would be the best practice — pythonicwiz
– pythonicwiz, Commented Jun 1, 2021 at 11:10

FMc · Accepted Answer · 2021-06-02 04:52:16Z

You have made a good decision to read the rows as dicts. You can simplify the code further by taking fuller advantage of the csv library's ability to write dicts as well.

It's a good habit to write data transformation programs like this with a separation between data collection, data conversion, and data output -- at least if feasible, given other important considerations. Such separation has many benefits related to testing, debugging, and flexibility in the face of evolving requirements. In addition, drawing clear boundaries tends to result in code that is easier to read and understand quickly.

Another good habit for such programs is to avoid hardcoding files paths in the script (other than as default values). It's often handy, for example, to test and debug with small files. Paths to those alternative files can come from command-line arguments.

If you want to be rigorous, you could extract a few more constants out of the code.

An illustration:


import csv
import sys

RENAMED_HEADERS = {
    'First_name': 'Name',
    'Age_Years':'Age',
    'Gender_F_M':'Gender',
    'Religion': None,
    'Nationality': 'Nation',
}

DELIMITER = ','

PATHS = ('input.csv', 'output.csv')

def main(args):
    input_path, output_path = args or PATHS
    rows = read_rows(input_path)
    converted = convert_rows(rows)
    write_rows(output_path, converted)

def read_rows(path):
    with open(path) as fh:
        reader = csv.DictReader(fh, delimiter=DELIMITER)
        return list(reader)

def convert_rows(rows):
    return [
        {
            new : r.get(old, '')
            for new, old in RENAMED_HEADERS.items()
        }
        for r in rows
    ]

def write_rows(path, rows):
    header = list(RENAMED_HEADERS)
    with open(path, 'w', newline='') as fh:
        writer = csv.DictWriter(fh, fieldnames=header, delimiter=DELIMITER)
        writer.writeheader()
        writer.writerows(rows)

if __name__ == '__main__':
    main(sys.argv[1:])

Stack Exchange Network

Mapping CSV columns

input.csv:

output.csv:

code:

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Mapping CSV columns

input.csv:

output.csv:

code:

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions