Python: Store command output in variable

Question

I'd like to measure the time and cpu usage of a command with the /usr/bin/time tool. But when I do os.popen( "/usr/bin/time -f \t%E MM:ss:mm ls -R" ).read() it also stores the output of ls -R. What can I do to only store the /usr/bin/time output?

I also tried it with subprocess but it doesn't work either.

out = subprocess.Popen(['/usr/bin/time', '-f', '"\t%E MM:ss:mm"', 'ls', '-R'], 
       stdout=subprocess.PIPE, 
       stderr=subprocess.STDOUT)
stdout,stderr = out.communicate()
print( stdout )

Running the command in the terminal looks like this:

Input:

/usr/bin/time -f "\t%E M:ss:mm, \t%P CPU" ls -R

Output:

.:
Dockerfile  input  output  README.md  src  Test.py

./input:
links.txt  raw  yuv

./input/raw:

./input/yuv:

./output:

./src:
InstallEncoderArm.sh  InstallEncoderx86.sh  RunTests.py
    0:00.00 M:ss:mm,    100% CPU

how would you do this in the console? If you weren't in Python — Adam Smith
– Adam Smith, Commented Jul 7, 2020 at 6:03
I added the I/O for the terminal command but Im not sure what you mean — René
– René, Commented Jul 7, 2020 at 6:10
You're simply calling terminal commands. If you weren't trying to pipe this through Python and just wanted to get the output you expect, what command would you write? — Adam Smith
– Adam Smith, Commented Jul 7, 2020 at 6:11

Technophobe01 · Accepted Answer · 2020-07-07 08:38:14Z

2

You need to send the output of the command to /dev/null

>/usr/bin/time -f "\t%E real,\t%U user,\t%S sys" ls -Fs >/dev/null
        0:00.01 real,   0.00 user,      0.01 sys

The complexity here is we want to throw away the command output but store the output of the /usr/bin/time command.

To store the output of usr/bin/time as a variable is a little more complicated as /usr/bin/time presents its output on stderr. So we need to send the command output to dev/null, then redirect the output of time from stderr and capture it in a variable. Assuming you may want to execute more complex commands than ls -R we would normally call sh -c 'exec ' this will give you more options in the future. Thus:

result=$(/usr/bin/time -f "\t%E MM:ss:mm" sh -c 'exec ls -R >/dev/null' 2>&1 tee)

Execution Output:

>result=$(/usr/bin/time -f "\t%E MM:ss:mm" sh -c 'exec ls -R >/dev/null' 2>&1 tee
); echo $result
0:20.60 MM:ss:mm

here we capture the result as an environment variable

>echo $result
0:20.60 MM:ss:mm

finally we arrive at:

os.popen("/usr/bin/time -f '%E MM:ss:mm' sh -c 'exec ls -R >/dev/null' 2>&1 tee").read()

Execution Out:

>python3
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.popen("/usr/bin/time -f '%E MM:ss:mm' sh -c 'exec ls -R >/dev/null' 2>&1 tee").read()
'0:19.89 MM:ss:mm\n'

hoping the above points you in the right direction.

edited Jul 7, 2020 at 8:38

answered Jul 7, 2020 at 6:14

Technophobe01

8,7663 gold badges38 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

René Over a year ago

Looks great in the terminal but how do I store it as a variable

SiHa Over a year ago

@René - how to you assign anything to a variable in Python? (Hint: it involves =)

René Over a year ago

I know how assigning works but when i do out = os.popen( "/usr/bin/time -f \t%E MM:ss:mm ls -R >/dev/null 2>&1" ).read() and after that print(out) it print a blank line

Technophobe01 Over a year ago

@René I updated the answer to address your question of how to assign the time output to a shell variable. Take care and stay safe.

Hadi Rahjoo · Accepted Answer · 2020-07-07 06:34:24Z

this code worked for me on python 2.7

import os
from datetime import datetime
CPU_Pct="Cpu Usage :"+str(round(float(os.popen('''grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage }' ''').readline()),2))+ "  time:" + datetim$

print(CPU_Pct)

the output will be like

5.74  time:2020-07-07 10:53:22

and if you wanted to get usage of memory too you can add this line to your code

tot_m, used_m, free_m = map(int, os.popen('free -t -m').readlines()[-1].split()[1:])

the final code could be this :

import os
from datetime import datetime
CPU="|| CPU Usage :"+str(round(float(os.popen('''grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage }' ''').readline()),2))
Time =  "||  time:" + datetime.now().strftime('%H:%M:%S')
tot_m, used_m, free_m = map(int, os.popen('free -t -m').readlines()[-1].split()[1:])
Memory ="|| memory :"+ str(used_m) +"/"+str(tot_m)
print(Time + CPU+Memory)

and here is the output :

||  time:11:02:33|| CPU Usage :5.74|| memory :13847/37529

René · Accepted Answer · 2020-07-07 07:07:06Z

0

It works with subprocess but notice that /usr/bin/time uses stderr

import subprocess

proc = subprocess.Popen(["/usr/bin/time -f \"\t%E M:ss:mm, \t%P CPU\" ls -R"], 
stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
print("program output:", err.decode("utf-8"))

Output:

program output:         0:00.00 M:ss:mm,        100% CPU

answered Jul 7, 2020 at 7:07

René

4083 silver badges13 bronze badges

Collectives™ on Stack Overflow

Python: Store command output in variable

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related