Import CSV to list of list Python

Question

Python newbie here.

I have a CSV file containing numbers in this kind of format

9143.680696, 427953.500000, 11919.104475, 11908.727555, 1.000871, 0.029506, 15.546608, 93, 121, 123, 7
7704.773182, 330297.500000, 19186.759308, 19170.146116, 1.000867, 0.029426, 14.302257, 93, 121, 123, 7

I need to read the file such that the list will go like this

[
[[9143.680696, 427953.500000, 11919.104475, 11908.727555, 1.000871, 0.029506, 15.546608, 93, 121, 123], [7]],
[[7704.773182, 330297.500000, 19186.759308, 19170.146116, 1.000867, 0.029426, 14.302257, 93, 121, 123], [7]]
]

The last number of every line is stored in a different list like in the case of 7 here.

I have researched some of the answers here but have found that they are stored as strings into the list which would not be compatible to the problem I am dealing with.

Thank you in advance for your help.

Do you expect the last four items of each line to be treated as ints or floats (93 or 93.0)? — RafG
– RafG, Commented Apr 27, 2016 at 7:44

Adem Öztaş · Accepted Answer · 2016-04-27 07:56:27Z

3

You can try like this,

>>> csv = '''9143.680696, 427953.500000, 11919.104475, 11908.727555, 
1.000871, 0.029506, 15.546608, 93, 121, 123, 7
7704.773182, 330297.500000, 19186.759308, 19170.146116, 1.000867, 0.029426, 14.302257, 93, 121, 123, 7'''
>>> [[line.split(',')[0:-1], [line.split(',')[-1]]] for line in csv.splitlines()] 
[[['9143.680696', ' 427953.500000', ' 11919.104475', ' 11908.727555', ' 1.000871', ' 0.029506', ' 15.546608', ' 93', ' 121', ' 123'], [' 7']], [['7704.773182', ' 330297.500000', ' 19186.759308', ' 19170.146116', ' 1.000867', ' 0.029426', ' 14.302257', ' 93', ' 121', ' 123'], [' 7']]]

If you want to float items, you can use map,

>>> data = csv.splitlines()
>>> data = [map(float, line.split(',')) for line in csv.splitlines()]
>>> [[items[:-1], items[-1]] for items in data]
[[[9143.680696, 427953.5, 11919.104475, 11908.727555, 1.000871, 0.029506, 15.546608, 93.0, 121.0, 123.0], 7.0], [[7704.773182, 330297.5, 19186.759308, 19170.146116, 1.000867, 0.029426, 14.302257, 93.0, 121.0, 123.0], 7.0]]

Pretty print:

>>> import pprint
>>> pprint.pprint([[items[:-1], items[-1]] for items in data])
[[[9143.680696,
   427953.5,
   11919.104475,
   11908.727555,
   1.000871,
   0.029506,
   15.546608,
   93.0,
   121.0,
   123.0],
  7.0],
 [[7704.773182,
   330297.5,
   19186.759308,
   19170.146116,
   1.000867,
   0.029426,
   14.302257,
   93.0,
   121.0,
   123.0],
  7.0]]

edited Apr 27, 2016 at 7:56

answered Apr 27, 2016 at 7:36

Adem Öztaş

21.6k5 gold badges36 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Whysmerhill Over a year ago

I think that ethanruan want to store decimal number so add the conversion ?

RafG Over a year ago

On Python 3 you would need list(map(...)) or a list comprehension.

Adem Öztaş Over a year ago

@Whysmerhill I've added a solution. Thanks

DNA · Accepted Answer · 2016-04-27 07:38:36Z

CSV libraries typically read the fields as strings, so you will need to convert the fields explicitly. From the documentation of the csv module:

Each row read from the csv file is returned as a list of strings. No automatic data type conversion is performed.

>>> import csv
>>> with open('eggs.csv', 'rb') as csvfile:
...     spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
...     for row in spamreader:
...         <process row>

Similarly, CSV libraries will treat all fields equally, so you will need to explicitly wrap the last field in a list.

For example:

a = ["1.23", "2.34", "10", "100", "1000"]

>>> map(float, a[0:2]) + map(int, a[2:4]) + [[int(a[4])]]

[1.23, 2.34, 10, 100, [1000]]

Byte Commander · Accepted Answer · 2016-04-27 09:58:08Z

0

The simplest approach without using external modules:

Update: I replaced the simple float(...) conversions with the new convert(...) method that tries to produce a float and returns the original string (or alternatively could do something else) instead of throwing an exception if the token is not a number.

def convert(value_str):
    try:  # try to convert it to a float:
        return float(value_str)
    except ValueError:  # if it is not a valid float literal, return the original string:
        return value_str

with open("file.csv") as csvfile:
    split_lines = [line.split(",") for line in csvfile]
    data = [[[convert(n) for n in line[:-1]], [convert(line[-1])]] for line in split_lines]

    print(data)

Output for your example data from the question (formatted manually):

[
  [ [9143.680696, 427953.5, 11919.104475, 11908.727555, 1.000871, 0.029506, 15.546608, 93.0, 121.0, 123.0], [7.0] ],
  [ [7704.773182, 330297.5, 19186.759308, 19170.146116, 1.000867, 0.029426, 14.302257, 93.0, 121.0, 123.0], [7.0] ]
]

edited Apr 27, 2016 at 9:58

answered Apr 27, 2016 at 7:45

Byte Commander

6,8467 gold badges48 silver badges77 bronze badges

3 Comments

ethanruan Over a year ago

Would it bring out errors if I have values in the list that are -nan?

Byte Commander Over a year ago

Yes, this assumes that there are only valid float literals separated by commas. You could add a check that e.g. returns the value as string instead if it fails to convert. Adding that to my answer...

Byte Commander Over a year ago

@ethanruan Added a convert() function that handles cases where the token is not a valid float number.

Sven Hakvoort · Accepted Answer · 2016-04-27 07:40:01Z

0

You need to iterate over the list and convert them to integers. Also store them in the list format you want.

example:

import csv
l = list()
with open('data.csv', 'r') as csvfile:
   reader = csv.reader(csvfile, delimiter=',')
   for row in reader:
   l.append([[float(row[:-1])]+[float(row[-1])]])
print(l)

answered Apr 27, 2016 at 7:40

Sven Hakvoort

3,6213 gold badges19 silver badges35 bronze badges

Comments

Zbynek Vyskovsky - kvr000 · Accepted Answer · 2016-04-27 08:08:24Z

0

Just use [] operator to get the left and right part of the list:

import csv
...
list = []
with open(filename, "rb") as fd:
    reader = csv.reader(fd, delimiter = ",")
    for row in reader:
        left = list(map(lambda x: float(x), row[:-1]))
        right = list(map(lambda x: float(x), row[-1:]))
        list.append([ left, right ])

edited Apr 27, 2016 at 8:08

answered Apr 27, 2016 at 7:38

Zbynek Vyskovsky - kvr000

19.1k3 gold badges41 silver badges49 bronze badges

Comments

Jithin · Accepted Answer · 2016-04-27 11:29:16Z

you can try this, consider the input file name as input.csv

import csv
new_list = []
with open('input.csv') as inp:
    csv_reader = csv.reader(inp, delimiter=',')
    for line in csv_reader:
        new_list.append([map(float, line[:-1])] + [map(float, line[-1:])])

demo from ipython,

In [1]: import csv

In [2]: new_list = []

In [3]: with open('input.csv') as inp:
   ...:     csv_reader = csv.reader(inp, delimiter=',')
   ...:     for line in csv_reader:
   ...:         new_list.append([line[:-1]] + [line[-1:]])
   ...:         

In [4]: new_list
Out[4]: 


    [[[9143.680696,
   427953.5,
   11919.104475,
   11908.727555,
   1.000871,
   0.029506,
   15.546608,
   93.0,
   121.0,
   123.0],
  [7.0]],
 [[7704.773182,
   330297.5,
   19186.759308,
   19170.146116,
   1.000867,
   0.029426,
   14.302257,
   93.0,
   121.0,
   123.0],
  [7.0]]]

Collectives™ on Stack Overflow

Import CSV to list of list Python

6 Answers 6

3 Comments

Comments

3 Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

3 Comments

Comments

3 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related