Reading and writing from and to different files in Python

Question

I am trying to read the commands from "command.txt" file and want to redirect the output of this commands to "output.txt", contents of command.txt

ps -a
free

So far I came up with this code which for certain reason is not good and fails to execute.

import os
import sys
import subprocess
with open('output.txt', 'w') as out_file, open('command.txt', 'r') as in_file:
     for line in in_file:
         output = subprocess.Popen(line, stdout=subprocess.PIPE)
         print output
         out_file.write(output)

I am getting the below error:

Error:
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7/Users/PythonTutorials/subprocess1.py
Traceback (most recent call last):
   File "/Users/shandeepkm/PythonTutorials/subprocess1.py", line 9, in <module>
   output = subprocess.Popen(line, stdout=subprocess.PIPE)
   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 709, in __init__
errread, errwrite)
   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1326, in _execute_child
raise child_exception
  OSError: [Errno 2] No such file or directory

Process finished with exit code 1

Could anyone please suggest appropriate python code for this task.

" I came up with this code which for certain reason is not good" -- which reason is that? "and fails to execute" -- how does it fail? — Brian Cain
– Brian Cain, Commented May 15, 2015 at 0:00
@ShandeepMurugasamy Ok, I found the answer to your updated problem. I can now run your code to completion. See my revised solution below. — chrisaycock
– chrisaycock, Commented May 15, 2015 at 0:20

chrisaycock · Accepted Answer · 2015-05-15 00:15:23Z

3

I see two errors.

First, you have "command.txt" as the first line in your file. That definitely won't execute as a subprocess.

Also, your line

out_file.write(output)

needs to be tabbed under the for loop.

Now for the updated question:

The Popen constructor needs to take an array for the args. So instead of

'ps -a'

you need to pass

['ps', '-a']

Also, what gets returned from Popen isn't text. So altogether you need:

args = shlex.split(line)
output = subprocess.Popen(args, stdout=subprocess.PIPE).stdout.read()

edited May 15, 2015 at 0:15

answered May 15, 2015 at 0:04

chrisaycock

38.1k15 gold badges94 silver badges128 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Shandeep Murugasamy Over a year ago

Thanks! I tried that but still getting the above errors.

chrisaycock Over a year ago

@ShandeepMurugasamy When you say you "tried that", you mean you removed "command.txt" from your file?

tenwest Over a year ago

On windows, I had to add shell = True. (Though this is due to me using specific windows shell commands, i.e. echo. If executing a batch file or the like, default False should be OK.

chrisaycock Over a year ago

@tenwest Good point. From the above link: "The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable."

Hai Vu · Accepted Answer · 2015-05-15 01:43:12Z

1

Solution 1: Execute line by line

You can redirect to a file using the stdout parameter of Popen:

import subprocess
import shlex

with open('output.txt', 'wb') as outfile, open('command.txt') as infile:
    for line in infile:
        command = shlex.split(line)
        if not command:
            continue  # Skip blank lines
        try:
            process = subprocess.Popen(command, stdout=outfile)
            process.wait()
        except OSError:
            outfile.write('COMMAND ERROR: {}'.format(line))

In the code above, you redirect the output by pointing stdout to the output file's handle, no printing is needed. The code also guard against bad commands

Solution 2: Call the shell

If you are running under Linux or Mac, the following solution is simpler: by calling bash to execute the whole command.txt file and record the stdout and stderr. This should work under windows with cmd -c, but I don't have a Windows machine to try.

import subprocess

with open('output.txt', 'wb') as outfile:
    command = ['bash', 'command.txt']
    process = subprocess.Popen(command, stdout=outfile, stderr=outfile)
    process.wait()

answered May 15, 2015 at 1:43

Hai Vu

41.4k16 gold badges75 silver badges106 bronze badges

2 Comments

Shandeep Murugasamy Over a year ago

So in the above code, command = shlex.split(line) would return ['ps', '-a'] and ['free']. And if in line process = subprocess.Popen(command, stdout=outfile, stderr=outfile), these 2 commands would be executed simultaneously right?

Hai Vu Over a year ago

Since we are reading the file one line at the time (first solution), we execute them one at a time, not simultaneously.

Collectives™ on Stack Overflow

Reading and writing from and to different files in Python

2 Answers 2

4 Comments

Solution 1: Execute line by line

Solution 2: Call the shell

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Solution 1: Execute line by line

Solution 2: Call the shell

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related