python subprocess print and store stdout

Question

I am learning python and writing a small script. I need my output to be in a file and also print the output on the screen as well.I tried various methods like stdout=subprocess.PIPEi could not figure out outputting to both.please forgive me for this silly question`

#!/usr/bin/python
import os
import subprocess
with  open('/root/test/testfile') as f , open('aaa.txt',"w") as final:
    content =  f.read().splitlines()
    for x in content:
            result= subprocess.Popen(['grep',x,'/news/today/night/logs'],stdout=final)

I think this is a duplicate. Let me know if it doesn't meet your needs. — tdelaney
– tdelaney, Commented Oct 23, 2020 at 5:43
@tdelaney i am unable to understand that solution comparing with mine question can you please assist me with the proc wait and sys command. — evanooruvan
– evanooruvan, Commented Oct 23, 2020 at 5:57
What are the lines in testfile? Some words to find in logs, or regular expressions? — tdelaney
– tdelaney, Commented Oct 23, 2020 at 6:06
I don't think you need a subprocess at all. Just have python find the lines. That's kinda what the existing answer does but it seems to have the files mixed up. i'll post my attempt. — tdelaney
– tdelaney, Commented Oct 23, 2020 at 6:07
@tdelaney thank you would be great if you could share your solution , i created something similar to your but i am getting only one grep output. — evanooruvan
– evanooruvan, Commented Oct 23, 2020 at 6:12

tdelaney · Accepted Answer · 2020-10-23 06:22:14Z

1

It looks like you are only using subprocess to run grep, but python can do grep-like matches of strings too. This program will read the lines in "testfile" and then write out lines from "log" that contain the testfile line. All of the log matches from the first line in "testfile" will be above the matches for the second line, etc... And log lines that match multiple testfile lines will output multiple times.

This code assumes that you are not matching a regular expression.

#!/usr/bin/python

# assuming logs is not too big, you could read it into memory once
# with open('/news/today/night/logs') as logfile:
#     logs = list(logfile)
    
with  open('/root/test/testfile') as f , open('aaa.txt',"w") as final:
    for wanted in f:
        wanted = wanted.strip()
        with open('/news/today/night/logs') as logs:
            for log in logs:
                if wanted in log:
                    print(log, end='')
                    final.write(log)

answered Oct 23, 2020 at 6:22

tdelaney

77.9k6 gold badges91 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

evanooruvan Over a year ago

wow did not realize i could write this way... is there any drawback of using my code vs yours and can you help me print stdout in my code. Thanks a lot

tdelaney Over a year ago

My answer that I originally linked shows you how to do that. Make your subprocess stdout a pipe, then have a loop read the pipe and write to multiple places. stackoverflow.com/questions/15535240/…

evanooruvan Over a year ago

thank you , but as you said i cannot open the logs(open('/news/today/night/logs') as logs) in a memory as it will system/application logs which can be big i'll give your other method a try and will update you :)

Cole Arseneau · Accepted Answer · 2020-10-23 06:17:47Z

0

Try this:

#!/usr/bin/python
import os

with open('/root/test/testfile') as f, open('/root/test/aaa.txt',"a") as final:
    content = f.read().splitlines()
    for line in content:
        if "/news/today/night/logs" in line:
            print(line)
            final.write(line)

I made your aaa.txt file append rather than write.

edited Oct 23, 2020 at 6:17

answered Oct 23, 2020 at 5:06

Cole Arseneau

1046 bronze badges

7 Comments

evanooruvan Over a year ago

sorry but i am getting x value printed i am trying to get the output of grep result= subprocess.Popen(['grep',x,'/news/today/night/logs'],stdout=final)

evanooruvan Over a year ago

getting this error --- Traceback (most recent call last): File "/usr/sbin/akash.py", line 7, in <module> if '/news/today/night/logs' in line: NameError: name 'line' is not defined

Cole Arseneau Over a year ago

Haha my bad, forgot it was called x. Should be good now!

evanooruvan Over a year ago

sorry missing output. Thanks

Cole Arseneau Over a year ago

Any error code? Also does the file contain /news/today/night/logs in any line?

|

Collectives™ on Stack Overflow

python subprocess print and store stdout

2 Answers 2

3 Comments

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related