2

I’m new to python and I’m having difficulties implementing a nested ‘for loop’. This might be simple but the following sample code which I tried doesn’t give me the intended result. My task actually is to read the records from an attribute table (ArcGIS feature data) and compare it with every record in a CSV file. But initially I’m trying to do the same on with 2 CSV files and then apply the similar logic to my original problem. I’m trying to figure out the working of the loop and I can add the conditions of comparison later on. Any help is greatly appreciated. Thanks.

The idea is that first row in file 1(CSV) compares itself to all the rows (row by row) in file 2(CSV), and then second row in file 1 does the same until each row of file 1 compares itself to all the rows in file 2. So in the anticipated outcome, I’m trying to see if for every row in file 1, if each row in file2 is considered.

Example:

**File 1   File 2**
ALPHA      All
BETA       Bell
GAMMA      Cell
DELTA      Dell
ITA

Sample code:

import csv, sys, os, string 
table1 = os.path.join(path, 'table1.csv')
table2 = os.path.join(path, 'table2.csv')
file1 = csv.reader(open(table1, 'r'))
file2 = csv.reader(open(table2, 'r'))
for row in file1:
    print row
    for prow in file2:
        print prow

Anticipated outcome:

   ALPHA
    All
    Bell
    Cell
    Dell

    BETA
    All 
    …..

    ITA

All
..
Dell
Improve this question
3
  • So, what's the question? Commented Jun 26, 2012 at 15:20
  • @ Marcin: I was trying to see how the loop should be formatted to achieve the desired result.the solution suggested by @ Jonas Wielicki worked. Commented Jun 26, 2012 at 16:50
  • @aglassman: when I tried, it would display the contents of file2 only once and then print the contents of file1 Commented Jun 26, 2012 at 16:52

5 Answers 5

Reset to default
3

The problem here is, that file2 is just a one-shot iterator. So after iterating over file2 once (in the first iteration of file1), you completely deplete the data.

Instead, you have to store the contents of file2 in a list:

file2=list(csv.reader(open(table2,'r')))
for row in file1:
     print row
     for prow in file2:
         print prow

This will print you some lists, all of which contain only one element, the first cell of the respective row. This is due to parsing the file as CSV. Each iteration gets you a list of cells in the rows.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

See my answer for an example. Also, I strongly suggest giving your examples as valid python.
@ Jonas Wielicki: Thank you. This gives me the intended result. I wasn't aware of one-shot iterator. I thought it would go over the data again each time the loop runs.
3

It's because as you iterate over a csv.reader object, it becomes empty with each iteration.

that's the reason that file2 iterator is behaving this way.

To get around this you should save the values from file2 in a list first.

file1=csv.reader(open(table1,’r’))
file2=list(csv.reader(open(table2, 'r'))) #edited this
for row in file1:
...     print row
...     for prow in file2:
...         print prow
Improve this answer

2 Comments

-1 A string would be much better, as it would eliminate the nested loop.
I think that list(csv.reader(open(table2, 'r'))) is more readable than list comprehension that essentially only copies iterator contents.
1

The problem is that after you iterate over all the rows of file2, its stream is consumed. There's simply nothing more to read. Next for loop will not re-set the csv.reader object, but rather recognise that everything was already read and parsed, and so there's nothing more to do.

Solutions might include:

file2_stream = open(table2, 'r')
for row in file1:
  print row
  file2_stream.seek(0)  # Reset file stream position
  file2 = csv.reader(file2_stream)  # Init CSV parsing
  for prow in file2:
    print prow

Or, you could ever reopen file each time:

for row in file1:
  print row
  file2 = csv.reader(open(table2, 'r'))
  for prow in file2:
    print prow

This, obviously will parse the second file at each outer iteration. If the file is not large in comparison to the memory size, you might want to parse it once, and then store result as a list in memory:

file2_rows = list(file2)
for row in file1:
  print row
  for prow in file2_rows:
    print prow
Improve this answer

Comments

1

Don't do this.

Read the first file into an appropriate datastructure (e.g. a set), then when reading the second file, test against the collected rows in the datastructure.

For this exercise, it might be best to create single string, as you seem to print the whole data read from the first file each time.

file2='\n'.join((l[0] for l in csv.reader(open(table2,’r’))))
for row in file1:
        print row
        print file2

If you need indenting, you can use textwrap to add indents to file2.

Improve this answer

7 Comments

A set is not appropriate, as it does not preserve ordering (which might be necessary). I.e. it removes information from the initial set of data.
@JonasWielicki For real work involving comparisons (which is what OP refers to), a set will almost certainly be what is wanted. For the exercise described, a string would be much better (it's the same thing every time).
Okay, agreed. Still, it's not the solution to the point of having a depleted iterato. I think you should at least mention the loss of order in your answer, as it's really an unexpected result in that specific loop if elements are mixed up.
@JonasWielicki It's exactly the solution to a depleted iterator. That's why iterators are used - to build up something else, if the data need to be accessed more than once.
No, at least not for the general case, because you're dropping information which was previously available (ordering). I do not argue that for doing comparisions later (which is what the OP seems to aim for) a set might be suitable. But this is not the general case.
|
1

The CSV module is going to return iterators for both of those files that will be "spent" after they are looped through. This is the typical Python behavior for files.

In order to use the values from one file in a loop for the other, you can load them into memory. Going off the best reading of your intention, I assume you want to associate the data in one file with every line in the other. I'll give an expository example:

greek = csv.reader(open('file1.csv'))
dells = csv.reader(open('file2.csv'))

second_file_data = list(dells)

#From here, dells is "spent."  If we would want to reuse it, we have to reopen it

for line in greek:
  print line
  for other in second_file_data:
      print other
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.