Escaping quotation marks in python string

Question

I'm using subprocess to call a program within python and I'm passing a string to it, which can contain quotation marks.

This is the piece of code that is giving me troubles

import subprocess
text = subprocess.Popen("""awk 'BEGIN { print "%s"}' | my_program """ % sentence, stdout=subprocess.PIPE, shell=True)

When sentence = "I'm doing this" I get the following error message

/bin/sh: -c: line 0: unexpected EOF while looking for matching `"'
/bin/sh: -c: line 1: syntax error: unexpected end of file

I guess this has to do with the way quotes are escaped in python and linux. Is there a way to fix it?

try sentence = "I\'m doing this". you're confusing awk because there's a quote in your quoted awk expression. — Jean-François Fabre
– Jean-François Fabre ♦, Commented Mar 27, 2017 at 9:34

Jean-François Fabre · Accepted Answer · 2017-03-27 11:24:59Z

1

you're confusing awk and underlying shell because there's a quote in your quoted awk expression. First part is equivalent to:

awk 'BEGIN { print "I'm doing this"}'

Which is incorrect, even in pure shell.

Quickfix, escape the quotes in your sentence:

text = subprocess.Popen("""awk 'BEGIN { print "%s"}' | my_program """ % sentence.replace("'","\\'"), stdout=subprocess.PIPE, shell=True)

Proper fix: don't use awk at all just to print something, just feed input to your subprocess:

text = subprocess.Popen(my_program, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output,error = text.communicate(sentence.encode())

(and you can get rid of the shell=True in the process)

Last point: you seem to have trouble because my_program is some program plus arguments. To pass a command such as aspell -a you can do:

my_program = "aspell -a"

or:

my_program = ['aspell','-a']

but not

my_program = ['aspell -a']

which is probably what you've done here, so Python tries to literally execute the program "aspell -a" instead of splitting into program + argument.

edited Mar 27, 2017 at 11:24

answered Mar 27, 2017 at 9:36

Jean-François Fabre♦

141k24 gold badges179 silver badges246 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Brian Over a year ago

Thanks so much for your answer, but I get this error when I try to replace my_program with what is should be ("aspell -a") FileNotFoundError: [Errno 2] No such file or directory: 'aspell -a'.

tripleee Over a year ago

You want to avoid a shell and run Popen(['aspell', 'a']) instead of putting the first argument as string. See also stackoverflow.com/questions/3172470/…

tripleee Over a year ago

Incidentally, avoiding these quoting issues is another good reason to avoid shell=True.

Jean-François Fabre Over a year ago

@tripleee actually it works when passing a string with spaces in it. Which doesn't work is passing a list of parameters but with incorrect splitting: ['aspell -a']

tripleee Over a year ago

Yeah, with the additional caveat that I guess the last one might actually work on Windows.

|

Collectives™ on Stack Overflow

Escaping quotation marks in python string

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related