How to execute a complicated external command from within python?

Question

I wonder how to execute a complicated external command from within python?

It works well in linux bash command line:
$ mysqldump --single-transaction -u myusername -pmypassword mydb mytable > mytable.sql

But it doesn't work from within python code:
subprocess.call(['mysqldump' '--single-transaction' '-u myusername' '-pmypasword' 'mydb' 'mytable' '>' 'mytable.sql'])

What's wrong with my python code?

Or I have to use os.system() ??

Please tell me how it can work from within python code, thanks in advance.

Pedro Castilho · Accepted Answer · 2017-03-23 02:10:44Z

1

The problem is that > isn't a command-line argument to the program. >, in this context, is actually a shell output redirection operator. It gets processed by the shell itself, not by the mysqldump program.

Python's subprocess knows nothing about the shell, so it can't understand shell operators such as >. It just tries to give > and mytable.sql to mysqldump as arguments, which mysqldump also does not understand, causing an error.

What you need to do is set up your subprocess to redirect its output. The way to do that in Python is:

with open('mytable.sql', 'w') as f:
  subprocess.call(['mysqldump',
                   '--single-transaction', 
                   '-u',
                   'myusername',
                   '-pmypasword', 
                   'mydb',
                   'mytable'], stdout=f)

The stdout argument to subprocess.call allows you to direct your subprocess's output wherever you like, just as the > shell operator does.

edited Mar 23, 2017 at 2:10

answered Mar 23, 2017 at 1:54

Pedro Castilho

10.6k3 gold badges30 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

adorp Over a year ago

Sir, your solution is totally correct, thank you from heart!

Pedro Castilho Over a year ago

I'm glad to help! If my answer has been helpful, please upvote and accept it :)

adorp Over a year ago

Hi Pedro, I tried upvoting quite a few times, but always ended up with failures, weird! Did I do something wrong?

adorp Over a year ago

Gotcha, stackoverflow says that my vote will be recorded but not displayed publicly because I am a newbie here, my reputation value is less than 15 points. INTERESTING!

northerntechie · Accepted Answer · 2017-03-23 01:57:25Z

1

It looks to be a syntax error,

subprocess.call(['mysqldump' '--single-transaction' '-u' 'myusername' '-pmypasword' 'mydb' 'mytable' '>' 'mytable.sql'])

Missing the '-u' 'myusername' quote separation.

Also, subprocess.run replaces subprocess.call in version >= 3.5. Check python subprocess.call API deprecation This should work for you.

answered Mar 23, 2017 at 1:57

northerntechie

113 bronze badges

2 Comments

adorp Over a year ago

No, after correcting this typo, it still doesn't work

northerntechie Over a year ago

You are right, it doesn't work, my error. Pedro is correct with the redirect to stdout. The deprecated subprocess.call and typo still applies.

yokawasa · Accepted Answer · 2017-03-23 02:15:11Z

0

Here is an example of executing complicated external command from within python:

https://github.com/yokawasa/azure-shell

Azure-shell is an iteractive Azure CLI 2.0 commandline interface that it internally us subprocess

import subprocess
full_cmd = "external command"
p = subprocess.Popen(full_cmd, shell=True, env=self._env)
p.communicate()

An external command here is az command (azure-cli 2.0) which has lots of commands options and parameters, which thus can be very complicated. Therefore, azure-shell support auto-completion of both command options and parameters for the azure-cli that helps its users to save time for checking azure cli command reference or typing 'az -h'. To achieve the auto-completion capabilities, it leverage python prompt toolkit

https://github.com/jonathanslenders/python-prompt-toolkit

Hope it would help

answered Mar 23, 2017 at 2:15

yokawasa

112 bronze badges

1 Comment

adorp Over a year ago

I don't want to import more complicated thing for this simple requirement, os.system() is much easier and it's a built-in function. Thanks anyway.

Collectives™ on Stack Overflow

How to execute a complicated external command from within python?

3 Answers 3

4 Comments

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related