Opening multiple files, creating lists, and dictionaries with for loop and range function in iPython

Question

I am trying to write a script that will take 10 data files, read them line by line making a dictionary of the first(gene) and third(value) item in the file, and then combine all of these into an output file with a 12th column showing the average among the 10 replicates. (I will be doing this for several different samples, each with 10 replicates.) I am trying to use a for loop and the range function to automatically generate the file names, dictionary names, etc. as I am pretty sure this is a cleaner way to do the job than writing out the the code to make the dictionary 10 times. However, I am new to coding and I keep getting a "SyntaxError: can't assign to function call" on line 11 of my code ('gene{0}key'.format(i) = []) and I am sure it will do the same for several other lines of code that are structured the same way. If someone knows what I am doing wrong I would love to know what it is! Thank you in advance!

#!/usr/bin/env ipython

import re
import itertools
import numpy

sample = raw_input('sample: ')

for i in range(10):
    filename = sample+'_rand{0}.genes.tajD'.format(i)
    'gene{0}key'.format(i) = []
    'taj{0}value'.format(i) = []
    with open(filename, 'r') as data:
        for line in data:
            line = line.strip().split('\t')
            gene, taj = line[0], line[3]
            'gene{0}key'.format(i).append(gene)
            'taj{0}value'.format(i).append(taj)
    'dic{0}'.format(i) = dict(itertools.izip('gene{0}key'.format(i),'taj{0}value'.format(i)))

outfilename = sample+'_tajD.genes.all'

with open(sample+'_rand0.genes.tajD', 'r') as genes, open(outfilename, 'w') as outfile :
    for line in genes :
        line = line.strip().split('\t')
        mastergene = line[0]
        for i in range(10):
            'value{0}'.format(i) = 'dic{0}'.format(i)[mastergene]
        allrand = [value0, value1, value2, value3, value4, value5, value6, value7, value8, value9]
        avg = numpy.mean(allrand)
        outfile.write(mastergene + '\t' + value0 + '\t' + value1 + '\t' + value2 +'\t' + value3 + '\t' + value4 + '\t' + value5 + '\t' + value6 + '\t' + value7 + '\t' + value8 + '\t' + value9 + '\t' + avg + '\n')

What do you want to get by this line 'dic{0}'.format(i) = dict(itertools.izip('gene{0}key'.format(i),'taj{0}value'.format(i)))? — akaRem
– akaRem, Commented Oct 4, 2013 at 18:22

Kevin · Accepted Answer · 2013-10-04 17:51:29Z

3

'gene{0}key'.format(i) = []

This is a typical antipattern that new coders attempt to use. You want to create ten variables with the names gene1key, gene2key, gene3key, etc. The correct way to do this is to use a list or dictionary, which is then indexed using i as the key. The simplest way to rewrite your code so it is syntactically correct is:

genekeys = {}
tajvalues = {}
dictionaries = {}

for i in range(10):
    filename = sample+'_rand{0}.genes.tajD'.format(i)
    genekeys[i] = []
    tajvalues[i] = []
    with open(filename, 'r') as data:
        for line in data:
            line = line.strip().split('\t')
            gene, taj = line[0], line[3]
            genekeys[i].append(gene)
            tajvalues[i].append(taj)
    dictionaries[i] = dict(itertools.izip(genekeys[i],tajvalues[i]))


with open(sample+'_rand0.genes.tajD', 'r') as genes, open(outfilename, 'w') as outfile :
    for line in genes :
        values = {}
        line = line.strip().split('\t')
        mastergene = line[0]
        for i in range(10):
            values[i] = dictionaries[i][mastergene]
        allrand = [values[0], values[1], values[2], values[3], values[4], values[5], values[6], values[7], values[8], values[9]]
        avg = numpy.mean(allrand)
        outfile.write(mastergene + '\t' + values[0] + '\t' + values[1] + '\t' + values[2] + '\t' + values[3] + '\t' + values[4] + '\t' + values[5] + '\t' + values[6] + '\t' + values[7] + '\t' + values[8] + '\t' + values[9] + '\t' + avg + '\n')

The code could be further simplified. For instance, Since genekeys and tajvalues are used only to create dictionaries, you can just populate that dict directly without needing to store the intermediary values. you can make values a list, so you don't have to explicitly address each one during numpy.mean and outfile.write.

dictionaries = {}
for i in range(10):
    filename = sample+'_rand{0}.genes.tajD'.format(i)
    dictionaries[i] = {}
    with open(filename, 'r') as data:
        for line in data:
            line = line.strip().split('\t')
            gene, taj = line[0], line[3]
            dictionaries[i][gene] = taj

with open(sample+'_rand0.genes.tajD', 'r') as genes, open(outfilename, 'w') as outfile :
    for line in genes :
        line = line.strip().split('\t')
        mastergene = line[0]
        values = []
        for i in range(10):
            values.append(dictionaries[i][mastergene])
        #alternative to the above for loop: values = [dictionaries[i][mastergene] for i in range 10]
        avg = numpy.mean(values)
        outfile.write(mastergene + '\t' + '\t'.join(values) + '\t' + avg + '\n')

edited Oct 4, 2013 at 17:51

answered Oct 4, 2013 at 17:43

Kevin

76.5k13 gold badges141 silver badges168 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

akaRem Over a year ago

Why use dictionaries = {} for keys from range(10) -> It's a list!

ONeillMB1 Over a year ago

Thank you! This works (with the exception of the avg which has to do with my files not the code). I really appreciate your explanations - clears up a lot of the issues I was having. Just to make sure I am understanding this - is "dictionaries" a dictionary of the indexed dictionaries? I found that by changing dictionaries = {} to dictionaries = [] (as suggested in the above comment) the code no longer works... Thanks again!

ONeillMB1 Over a year ago

Thank you again! I just revised this code for a new purpose. Your explanation was so helpful that I was able to make modifications for a different project.

Collectives™ on Stack Overflow

Opening multiple files, creating lists, and dictionaries with for loop and range function in iPython

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related