Perl is not fully executing when called from Python script

Question

I have a Python script, which calls a Perl script that parses some files and produces output files. Again these output files are uploaded into MySQL database using the Python script. The Perl script takes around 5-7 mins to parse some files and produce the output files that the Python uses to upload.

The Python script is like:

import subprocess
pipe = subprocess.Popen(["perl", "./parser.pl"], stdin=subprocess.PIPE)
pipe.stdin.close()

#Now, load the output files from parser.pl to database.

sql01 = """LOAD DATA LOCAL INFILE 'sequence.parsed' INTO TABLE nasequenceimp FIELDS TERMINATED BY '\t' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n';"""

try:
  c.execute(sql01)
  conn.commit()
except StandardError, e:
  print e
  conn.rollback()

But, it seems that Python script is terminated and the Perl script continues to parse the files. So, Python shoots error sequence.parsed does not exist

Why is the Python script not waiting for the Perl to complete executing?

@mpapec, what was the scope of adding my Perl code in this question. Did you read my question before voting it down? — aki2all
– aki2all, Commented Oct 8, 2013 at 6:09

Logan Ding · Accepted Answer · 2013-10-08 05:39:57Z

3

The parent process doesn't wait for the child process spawned by subprocess.Popen by default. You should call:

pipe.wait()

after subprocess.Popen. It will wait the child process to terminate and let your python script continue.

Or you can just use os.system() instead.

edited Oct 8, 2013 at 5:39

answered Oct 8, 2013 at 5:30

Logan Ding

1,7711 gold badge12 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

aki2all Over a year ago

Should I have to put pipe.stdin.close() after pipe.wait()?

Logan Ding Over a year ago

@aki2all pipe.wait() will block your python script so you should put it after pipe.stdin.close().

Collectives™ on Stack Overflow

Perl is not fully executing when called from Python script

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related