Optimize the for loop in python

Question

Good, I currently have the following code:

n = 0
with open('/home/user/test.filter') as f:

    lines = f.readlines()

    for l in lines:
        if lines[n].startswith('-A vlan_8'):
            if "-o bond1.8" in lines[n]:
                    f = open('vlan8.filter_in', 'a')
                    f.write(l)
            else:
                f = open('vlan8.filter_out', 'a')
                f.write(l)
        if lines[n].startswith('-A vlan_10'):
            if "-o bond1.10" in lines[n]:
                f = open('vlan10.filter_in', 'a')
                f.write(l)
            else:
                f = open('vlan10.filter_out', 'a')
                f.write(l)
        if lines[n].startswith('-A vlan_15'):
            if "-o bond1.15" in lines[n]:
                f = open('vlan15.filter_in', 'a')
                f.write(l)
            else:
                f = open('vlan15.filter_out', 'a')
                f.write(l)

        # [...]

        n = n + 1

I thought about optimizing it with some accumulator or list to not make the script so extensive. Any suggestions?

You mean optimizing (faster) or refactoring (less repetition) ? — polku
– polku, Commented Jun 7, 2017 at 9:16
First, don't do == True - it is redundant; if the expression on the left side of == is equal to True, then, well, it's true. Second, as you have very similar code repeated multiple times, consider making it a function. Third, open your output files once, at the beginning, rather than opening them every time you need to write to them - you can keep a list, or better, a dict of them. — Błotosmętek
– Błotosmętek, Commented Jun 7, 2017 at 9:19
You could only open the files (e.g. vlan8.filter_in) once before the loop, not for every line that you process. Also, all the if clauses immediately under the for loop (except the first of course) could be turned into elifs. if lines[n].startswith is incorrect, it should be if lines.startswith, but I didn't edit them to avoid changing the semantics of the question. — CristiFati
– CristiFati, Commented Jun 7, 2017 at 9:19

cs95 · Accepted Answer · 2017-06-07 09:13:12Z

2

Sure you can. Maintain a list of these numbers as so:

numList = [8, 10, 15, ...]

Now, all you need is a little string formatting.

with open('/home/user/test.filter') as f:
    lines = f.readlines()

for i, l in enumerate(lines): # I've used enumerate to eliminate n but really you don't need it
    for num in numList:
        if l.startswith('-A vlan_%d' %num):
            if "-o bond1.%d" %num in l:
                f = open('vlan%d.filter_in' %num, 'a')
            else:
                f = open('vlan%d.filter_out' %num, 'a')

            f.write(l)
            f.close()
            break

answered Jun 7, 2017 at 9:13

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Paco H. · Accepted Answer · 2017-06-07 09:25:55Z

I think you want to make the code cleaner, not faster. If that is so maybe this would work:

import re

parameter_re = re.compile(r'^-A vla_(\d+).*')

with open('data.csv') as f:
    lines = f.readlines()

for line in lines:
    # Match required parameter
    match = parameter_re.match(line)

    # Skip line if it doesn't match the required parameter
    if not match:
        continue

    # Extract number from parameter
    number = int(match.group(1))

    # Create output parameter string
    output_str = '-o bond1.%d' % number

    # Select correct filename to save the line
    if output_str in line:
        output_filename = 'vlan%d.filter_in' % number
    else:
        output_filename = 'vlan%d.filter_out' % number

    # Write the line to the correct file
    with open(output_filename, 'a') as f:
        f.write(line)

And if you want to make it shorter (which I don't recommend, better for it to be readable):

import re

with open('data.csv') as f:
    lines = f.readlines()

for line in lines:
    match = re.match(r'^-A vla_(\d+).*', line)
    if not match:
        continue
    number = int(match.group(1))
    if '-o bond1.%d' % number in line:
        output_filename = 'vlan%d.filter_in' % number
    else:
        output_filename = 'vlan%d.filter_out' % number
    with open(output_filename, 'a') as f:
        f.write(line)

Collectives™ on Stack Overflow

Optimize the for loop in python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related