python subprocess.call() not executing in loop

Question

I have the following piece of code that generates plots with gnuplot:

import sys, glob, subprocess, os, time
for file in glob.glob('comb_*.out'):
    fNameParts = file[5:].split('.')[0].split('-')
    gPlotCmd = []
    gPlotCmd = 'unset border; set xl "rotation period #"; set yl "T [K]"\n'
    gPlotCmd += 'set key above\n'
    gPlotCmd += 'set grid xtics ytics\n'
    gPlotCmd += 'set term post eps enh color solid\n'
    gPlotCmd += 'set xrange [20:876]\n'
    gPlotCmd += 'set output "fig_TestRelax-' + fNameParts[1] + '-' + fNameParts[2] + '-' + fNameParts[3]  + '-' + fNameParts[4] + '.eps"\n'
    conf = fNameParts[1] + '-' + fNameParts[2] + '-' + fNameParts[3]
    gPlotCmd += 'plot "' + file + '" using ($1/36000):($9-$3) w l lw 5 title "OP3-OP1 ' + conf + '", "' + file + '" using ($1/36000):($6-$3) w l lw 3 title "OP2-OP1 ' + conf + '", "' + file + '" using ($1/36000):($9-$6) w l lw 1 title "OP3-OP2 ' + conf + '"'
    fw = open('temp.plt','w+')
    fw.writelines(gPlotCmd)
    subprocess.call(["gnuplot","temp.plt"])
    print(file)

In the last loop execution the subprocess.call(["gnuplot","temp.plt"]) does not seem to be executed. At the end of the program, temp.plt exists with data from the last iteration. Also print(file) is executed during the last loop. Also if I plot the temp.plt left after the program I get the last plot (so there is no problem on the side of the data). Only the line subprocess.call(["gnuplot","temp.plt"]) is not executed. I also tried popen and monitor stdout and stderr but both were empty (as in all other iterations. The checked the problem occurs both on linux and windows and in versions 3.3.5 and 2.7.3.

I conclude that there is something wrong with the script but I do not know what.

jfs · Accepted Answer · 2014-03-30 13:00:30Z

4

@lvc's and yours answer are correct; it is a buffering issue and fw.flush() should fix it. But you don't need the temporary file, you could pass the input commands to gnuplot directly without writing them to disk:

from subprocess import Popen, PIPE

p = Popen('gnuplot', stdin=PIPE)
p.communicate(input=gPlotCmd.encode('ascii'))

answered Mar 30, 2014 at 13:00

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

lvc · Accepted Answer · 2014-03-30 11:27:13Z

3

One likely error here is that the file temp.plt isn't actually written to disk at the time you run gnuplot. Python doesn't necessarily flush its buffers immediately after a call to writelines. This would mean that when gnuplot is launched from your script, it sees an empty file. It doesn't give an error, because an empty input isn't an error, and it has no way of knowing its expecting anything else. By the time you run it outside of your script, Python has exited and hence can't be holding anything in its own buffers anymore.

Use the with statement to ensure that fw is closed once you're done with it:

with open('temp.plt', 'w') as fw:
    fw.writelines(gPlotCmd)

subprocess.call(["gnuplot","temp.plt"])

answered Mar 30, 2014 at 11:27

lvc

35.2k10 gold badges76 silver badges100 bronze badges

Comments

atapaka · Accepted Answer · 2014-03-30 11:55:41Z

1

It seems I figured that out. I am missing fw.close(). The last lines of code should look:

fw = open('temp.plt','w+')
fw.writelines(gPlotCmd)
fw.close()
subprocess.call(["gnuplot","temp.plt"])
print(file)

Now the code produces the intended plots.

answered Mar 30, 2014 at 11:55

atapaka

1,4004 gold badges18 silver badges37 bronze badges

1 Comment

atapaka Over a year ago

lvc thanks, I did not reload the window and did not see your answer, but the solution with close should be ok?

Collectives™ on Stack Overflow

python subprocess.call() not executing in loop

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related