Python: Using str.split and getting list index out of range

Question

I just started using python and am trying to convert some of my R code into python. The task is relatively simple; I have many csv file with a variable name (in this case cell lines) and values ( IC50's). I need to pull out all variables and their values shared in common among all files. Some of these files share the save variables but are formatted differently. For example in some files a variable is just "Cell_line" and in others it is MEL:Cell_line. So first things first to make a direct string comparison I need to format them the same and hence am trying ti use str.split() to do so. There is probably a much better way to do this but for now I am using the following code:

import csv
import os
# Change working directory
os.chdir("/Users/joshuamannheimer/downloads")
file_name="NCI60_Bleomycin.csv" 
with open(file_name) as csvfile:
    NCI_data=csv.reader(csvfile, delimiter=',')
    alldata={}
    for row in NCI_data:
        name_str=row[0]
        splt=name_str.split(':')
        n_name=splt[1]
        alldata[n_name]=row

[1] name_str.split return a list of length 2. Since the portion I want is after the ":" I want the second element which should be indexed as splt[1] as splt[0] is the first in python. However when I run the code I get this error message "IndexError: list index out of range" I'm trying the second element out of a list of length 2 thus I have no idea why it is out of range. Any help or suggestions would be appreciated.

It should work for "MEL:Cell_line"; it will fail on "Cell_line", as it will only have splt[0]. You can use splt[-1] to always get the last element, however many there are. — Amadan
– Amadan, Commented Aug 7, 2015 at 1:19

Anand S Kumar · Accepted Answer · 2015-08-07 01:21:25Z

3

I am pretty sure that there are some rows where name_str does not have a : in them. From your own example if the name_str is Cell_line it would fail.

If you are sure that there would only be 1 : in name_str (at max) , or if there are multiple : you want to select the last one, instead of splt[1] , you should use - splt[-1] . -1 index would take the last element in the list (unless its empty) .

answered Aug 7, 2015 at 1:21

Anand S Kumar

91.4k18 gold badges196 silver badges179 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Paul · Accepted Answer · 2015-08-07 01:41:42Z

The simple answer is that sometimes the data isn't following the specification being assumed when you write this code (i.e. that there is a colon and two fields).

The easiest way to deal with this is to add an if block if len(splot)==2: and do the subsequent lines within that block.

Optionally, add an else: and print the lines that are not so spec or save them somewhere so you can diagnose.

Like this:

import csv
import os
# Change working directory
os.chdir("/Users/joshuamannheimer/downloads")
file_name="NCI60_Bleomycin.csv" 
with open(file_name) as csvfile:
    NCI_data=csv.reader(csvfile, delimiter=',')
    alldata={}
    for row in NCI_data:
        name_str=row[0]
        splt=name_str.split(':')
        if len(splt)==2: 
             n_name=splt[1]
             alldata[n_name]=row
        else:
             print "invalid name: "+name_str

Alternatively, you can use try/except, which in this case is a bit more robust because we can handle IndexError anywhere, in either row[0] or in split[1], with the one exception handler, and we don't have to specify that the length of the : split field should be 2.

In addition we could explicitly check that there actually is a : before splitting, and assign the name appropriately.

import csv
import os
# Change working directory
os.chdir("/Users/joshuamannheimer/downloads")
file_name="NCI60_Bleomycin.csv" 
with open(file_name) as csvfile:
    NCI_data=csv.reader(csvfile, delimiter=',')
    alldata={}
    for row in NCI_data:
        try:
            name_str=row[0]
            if ':' in name_str:
                splt=name_str.split(':')
                n_name=splt[1]
            else:
                n_name = name_str
            alldata[n_name]=row
        except IndexError: 
            print "bad row:"+str(row)

So, thanks I made a stupid mistake and using your guys help realized that the first row was the column headers and thus did not have the ":" format. I assumed that because it already worked in R everything was ok, but forgot that in R you can specify column headers when loading so they are not treated as variables.

Collectives™ on Stack Overflow

Python: Using str.split and getting list index out of range

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related