How do I convert this bash loop to python?

Question

How would I do file reading loop in python? I'm trying to convert my bash script to python but have never written python before. FYI, the reason I am read reading the file after a successful command competition is to make sure it reads the most recent edit (say if the URLs were reordered). Thanks!

#!/bin/bash

    FILE=$1

    declare -A SUCCESS=()
    declare -A FAILED=()
for (( ;; )); do
    # find a new link

    cat "$FILE" > temp.txt

    HASNEW=false

    while read; do
        [[ -z $REPLY || -n ${SUCCESS[$REPLY]} || -n ${FAILED[$REPLY]} ]] && continue
        HASNEW=true
        break
    done < temp.txt

    [[ $HASNEW = true ]] || break

    # download
    if axel --alternate --num-connections=6 "$REPLY"; then
        echo
        echo "Succeeded at $DATETIME downloading following link $REPLY"
        echo "$DATETIME Finished: $REPLY" >> downloaded-links.txt
        echo
        SUCCESS[$REPLY]=.
    else
        echo
        echo "Failed at $DATETIME to download following link $REPLY"
        echo "$DATETIME Failed: $REPLY" >> failed-links.txt


        FAILED[$REPLY]=.
    fi

    # refresh file

    cat "$FILE" > temp.txt

    while read; do
        [[ -z ${SUCCESS[REPLY]} ]] && echo "$REPLY"
    done < temp.txt > "$FILE"
done

This is what I've got so far which is working, and I can't figure out how to make it read the top line of the file after every successful execution of the axel line like the bash script does. I'm open to other options on the subprocess call such as threading, but I'm not sure how to make that work.

#!/usr/bin/env python
import subprocess
from optparse import OptionParser

# create command line variables
axel = "axel --alternate --num-connections=6 "

usage = "usage: %prog [options] ListFile.txt"
parser = OptionParser(usage=usage)
parser.add_option("-s", "--speed", dest="speed",
    help="speed in bits per second i.e. 51200 is 50kps", metavar="speedkbps")

(opts, args) = parser.parse_args()

if args[0] is None:
    print "No list file given\n"
    parser.print_help()
    exit(-1)

list_file_1 = args[0]

try:
    opts.speed
except NoSpeed:
    with open(list_file_1, 'r+') as f:
        for line in f:
            axel_call = axel + "--max-speed=" + opts.speed + " " + line
#           print ("speed option set line send to subprocess is: " + axel_call)
            subprocess.call(axel_call, shell=True)
else:
    with open(list_file_1, 'r+') as f:
        for line in f:
            axel_call = axel + line
#           print ("no speed option set line send to subprocess is:" + axel_call)
            subprocess.call(axel_call, shell=True)

what have you tried so far? It takes much less effort to improve or fix an already existing approach than to write one from scretch... — Niklas B.
– Niklas B., Commented Feb 28, 2012 at 3:01
What does that bash code even do? Why is it doing cat "$FILE" > temp.txt and then doing < temp.txt when you could just use < "$FILE" instead? — Niklas B.
– Niklas B., Commented Feb 28, 2012 at 3:07

Srikar Appalaraju · Accepted Answer · 2012-02-28 03:02:36Z

3

Fully Pythonic way to read a file is the following:

with open(...) as f:
    for line in f:
        <do something with line>

The with statement handles opening and closing the file, including if an exception is raised in the inner block. The for line in f treats the file object f as an iterable, which automatically uses buffered IO and memory management so you don't have to worry about large files.

There should be one -- and preferably only one -- obvious way to do it.

answered Feb 28, 2012 at 3:02

Srikar Appalaraju

74k55 gold badges221 silver badges265 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Rich Over a year ago

Does with work with a suprocess call? i.e. with open(list_file_1, 'r+') as f: for line in f: subprocess.call(axel, list_file_1, shell=True) (How the heck to you get stackoverflow to respect whitespace in code withing a comment?)

Srikar Appalaraju Over a year ago

Is there any specific reason you want to spawn a subprocess? you could make a subprocess call but in such instances I find using threads to be a better option. That way later on to collate data from different sub-processes becomes a pain whereas threads being in the same process space is more elegant.

imm · Accepted Answer · 2012-02-28 03:04:57Z

0

A demonstration of using a loop to read a file is shown at http://docs.python.org/tutorial/inputoutput.html#reading-and-writing-files

answered Feb 28, 2012 at 3:04

imm

5,9391 gold badge29 silver badges32 bronze badges

Collectives™ on Stack Overflow

How do I convert this bash loop to python?

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related