Python upper limit for loop

Question

Let me start by saying I'm fairly new to python.

Ok so I'm running code to perform physics calculations/draw graphs etc on data files, and what I need to do is loop over a number of files and sub-files. But the problem is there are a different number of sub-files in each file (e.g. file 0 has 711 sub-files, file 1 has 660 odd). It obviously doesn't like it when I run it across a file that doesn't have a sub-file at index x, so I was wondering is there a way to get it to run (iterate?) up to the final limit in each file automatically?

What I've got is a nested loop like:

for i in range(0,120):
    for j in range(0,715):

         stuff

Cheers in advance for any help, and sorry if my explanation is bad!

Edit: some more of the code. So what I'm actually doing is calculating/plotting the angular momentum of gas and dark matter particles. These are in halos (j), & there are a number of files (i) containing lots and lots of these halos.

import getfiby
import numpy as np 
import pylab as pl


angmom=getfiby.ReadFIBY("B")

for i in range(0,120):
    for j in range(0,715):

        pos_DM = angmom.getParticleField(49,i,j,"DM","Coordinates")
        vel_DM = angmom.getParticleField(49,i,j,"DM","Velocity")
        mass_DM = angmom.getParticleField(49,i,j,"DM","Mass")
        more stuff

getfiby is a code I was given that retrieves all the data from the files (that I can't see). It's not really a massive problem, as I can still get it to run the calculations & make plots even if the upper limit I put on my range for j exceeds the number of halos there are in a particular file (I get: Halo index out of bounds. Goodbye.) But yeah I just wondered if there was a nicer, tidier way of getting it to run.

It would help if we could see a bit more of the code for context. — Joel Cornett
– Joel Cornett, Commented Feb 27, 2014 at 15:34
What`s a sub file? Are you asking how to iterate of the files in a directory and its subdirectories? — ikegami
– ikegami, Commented Feb 27, 2014 at 15:37
Are the files folders or ZIPs or some other format of files? — Ramchandra Apte
– Ramchandra Apte, Commented Feb 27, 2014 at 15:37
As a side note, changing those range to xrange will perform better under python2. If it's python3, leave it as is. — Daenyth
– Daenyth, Commented Feb 27, 2014 at 15:38

zhangxaochen · Accepted Answer · 2014-02-27 15:42:40Z

1

You may want something like this:

a=range(3)
for i in range(5):
    try:
        b=a[i] #if you iterate over your "subfiles" by index
    except IndexError:
        break #break out when list index out of range

answered Feb 27, 2014 at 15:42

zhangxaochen

34.2k15 gold badges82 silver badges115 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

sabbahillel · Accepted Answer · 2014-02-27 15:45:17Z

0

When you build the list of files and subfiles to process, you could have the list in a variable named filelist and the subfile list in a variable called subfilelist You could then run the loops as

for myfile in filelist:
  # Process code
  for subfile in subfilelist:
    # Process code.

If you need to use a range then

filerange = len(filelist)
subfilerange = len(subfilelist)

for i in range(0, filerange):
  for j in range(0, subfilerange):

answered Feb 27, 2014 at 15:45

sabbahillel

4,4351 gold badge22 silver badges37 bronze badges

Comments

Henry Keiter · Accepted Answer · 2014-02-27 15:52:49Z

I may be misunderstanding your structure/intent here, but it sounds like you want to perform some calculations on each file in some folder structure, where the folders contain various numbers of the data files. In that case, I'd make use of the os.listdir function rather than trying to manually index each data file.

import os
for file in os.listdir(ROOT_DIRECTORY):
    for subfile in os.listdir(os.path.join(ROOT_DIRECTORY, file)):
        process(os.path.join(ROOT_DIRECTORY, file, subfile)) # or whatever

This can of course be made a little easier to look at in a few ways (personally I have my own listdir wrapper that returns full paths instead of just the basenames), but that would be my basic idea.

If you also need the index of each file, you can probably get it using some combination of enumerate and maybe sorted (e.g. for i, file in enumerate(sorted(os.listdir(...)))), but the specifics obviously depend on your filenames and directory structure.

Emilio M Bumachar · Accepted Answer · 2014-02-27 15:56:11Z

0

Assuming fileSet is an iterable full of file objects, and each file object is itself an iterable full of subfile objects, than:

    for i,file in enumerate(fileSet): 
        for j,subfile in enumerate(file):
            stuff

But think hard about whether you really need the indices i and j, as you already have the words file and subfile to refer to the objects you are dealing with. If you don't need the indices, it's simply:

    for file in fileSet: 
        for subfile in file:
            stuff

Now, if by "file" you actually meant files in the Operating System's Filesystem, then I need you to explain me what a subfile is in that context, as Filesystem files usually cannot be nested, only directories can.

answered Feb 27, 2014 at 15:56

Emilio M Bumachar

2,6134 gold badges27 silver badges31 bronze badges

Collectives™ on Stack Overflow

Python upper limit for loop

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related