5

I am running script to unrar some files and the remove the rar files afterwards. I am doing this by running the command through shell. I have tried several different ways to make the script wait until it's done unpacking the files, but it still goes ahead and deletes the file before they are done being used.

I have tried the code below, which is a no go. I have tried to see if i could get the wait() to work, but also no luck.

Any ideas? running python 2.7

EDIT: I want the script to run the command :)

            p = subprocess.Popen('unrar e ' + root + '/' + i + ' ' + testfolder,
                                 bufsize=2048, shell=True,
                                 stdin=subprocess.PIPE)
            p.stdin.write('e')
            p.communicate()

for root, dirs, files in os.walk(testfolder):
    for i in files:

        print 'Deleting rar files'
        os.remove(i)

for i in os.listdir(testfolder):
    if os.path.isdir(testfolder + i):
        shutil.rmtree(testfolder + i)
Improve this question
4
  • 1
    if you wait for the command, you don't need subprocess Commented Apr 24, 2013 at 15:57
  • I would like to make my script run the command. Commented Apr 24, 2013 at 16:00
  • os.system("some command") is blocking afaik .... so it should block until unrar returns at least (which I assume is when it is finished) ... although i would think subproccess.communicate is also blocking ... my guess is that unrar does not wait until its finished to return an exit code Commented Apr 24, 2013 at 16:00
  • @KarolyHorvath However, using subprocess.Popen is far safer than using os.system(), as it's capable of being used without a shell. (Granted, it's not being used that way HERE, but that's bad practice and unsafe). Commented Apr 24, 2013 at 16:10

2 Answers 2

Reset to default
6

This is evil:

p = subprocess.Popen('unrar e ' + root + '/' + i + ' ' + testfolder,
        bufsize=2048, shell=True, stdin=subprocess.PIPE)

Instead, 

p = subprocess.Popen(['unrar', 'e', '%s/%s' % (root, i), testfolder],
        bufsize=2048, stdin=subprocess.PIPE)
p.stdin.write('e')
p.wait()
if p.returncode == 0:
    pass # put code that must only run if successful here.

By passing an exact array rather than a string to Popen and not using shell=True, a filename with a space in it can't be interpreted as a more than one arguments, or a subshell command, or some other potentially malicious thing (think of a file with $(rm -rf ..) in its name).

Then, after calling p.wait() (there's no need for p.communicate() when you aren't capturing stderr or stdout), you must check p.returncode to determine whether the process was successful, and only proceed on to delete files if p.returncode == 0 (indicating success).

Your initial diagnosis, that p.communicate() is returning while the unrar process is still running, is not feasible; p.communicate() and p.wait() do not work that way.

If running across ssh, this changes a bit:

import pipes # in Python 2.x; in 3.x, use shlex.quote() instead
p = subprocess.Popen(['ssh', ' '.join(
      [pipes.quote(s) for s in ['unrar', 'e', '%s/%s' % (root, i), testfolder]])
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

You may need p.stdin.close() before p.wait(), otherwise if your program waits for EOF your code will hang waiting for the sub-process.
Also, if the sub-process is ssh, I am seeing that even using the second form I still have to add quotes around arguments with spaces or else the remote system doesn't see them as a single argument.
@Michael, ssh does a remote round of expansion in addition to the local expansion that using this technique correctly avoids. If you want to prepare commands for ssh, use pipes.quote() (or, in Python 3.x, shlex.quote()) to escape your arguments.
@Michael, ...I've expanded the answer to directly address that scenario.
1

Is your problem waiting on the subprocess, or doing the things in order (meaning unpacking, then deleting).

If your problem is waiting on the subprocess, then you should check out the function subprocess.call

Check:

http://docs.python.org/2/library/subprocess.html#module-subprocess

That function blocks until the other process terminates.

If your problem however is unpacking the files, and you don't necesarrily have to use subprocessess, then just check out any other lib for unpacking, like pyunrar2:

or this other one:

Improve this answer

3 Comments

My problem is i want it to finish unpacking first and then delete the files. right now it moves on to delete the files before it can finish unpacking.
then just use subprocess.call() instead of subprocess.Popen
@vlad-ardelean Using subprocess.call() is incompatible with needing to write a character to the command's stdin, as the original code does.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.