3

I have a function which inside lauches multiple threads running in parallel. Threads print something and I want to capture this output from external function. I tried the next code to capture the output:

import sys, io
stdout = sys.stdout
sys.stdout = io.StringIO()
threads_conn(connect, devices) #- here many threads starts with many print inside
output = sys.stdout.getvalue()
sys.stdout = stdout

print(output)

This code works. But the problem is that output is printed only when all theads complete. So I have a freeze. Once all threads are done - whole output is printed. What I would like to have - have output once it is generated inside - in other words in real-time. Now I have the whole outout buffer printed at once. How get all of the threads to output in real-time?

3
  • 1
    Are you asking how to capture all the output into a string and also have it printed in real-time? Commented Jul 25, 2018 at 21:21
  • I guess you don't care that the output from the different threads gets all mixed together. Or would you prefer it if there is a way to identify the output of each thread? Commented Jul 25, 2018 at 21:23
  • Yes, I dont care. All I need to capture all output as it is generated. I need it to show it in my GUI in real time. Now GUI shows nothing when threads are working. Commented Jul 25, 2018 at 21:30

1 Answer 1

2

What you want to do is write your own TextIOBase class (or, if you want binary data, write your own RawIOBase and then wrap a stock TextIOWrapper around it), where you can put whatever behavior you want.

As you can see from the docs. all you need to implement for TextIOBase is detach, read, readline, and write. And the first three aren't relevant for what you're doing.


So, what should your write look like? Well, it depends on what you want to do.

it sounds like your goal is to tee everything to both real stdout and to a StringIO. If so, this is pretty trivial.

The only question is what you want to do if one of the targets raises an exception, or writes fewer bytes than the other, etc. Since an IOString is never going to do either of those, we can write something really dumb that just assumes that whatever real stdout did was the right thing to do.

class TeeTextIO(io.TextIOBase):
    def __init__(self, target):
        self.target = target
        self.stringio = io.StringIO()
    def write(self, s):
        writecount = self.target.write(s)
        self.stringio.write(s[:writecount])
        return writecount

And now:

stdout = sys.stdout
sys.stdout = TeeTextIO(sys.stdout)
threads_conn(connect, devices) #- here many threads starts with many print inside
output = sys.stdout.stringio.getvalue()
sys.stdout = stdout

Now, the output has gone to the real stdout as it came in, but it's also been stored in the StringIO for whatever you want to do with it later.

(Notice that this class will work with any TextIOBase, like a file you open, not just with stdout. It didn't cost us anything to make it general, so why not?)


What if you wanted to do something totally different, like spread each write randomly among 10 different files? It should be obvious:

class SpreadTextWriter(io.TextIOBase):
    def __init__(self, *files):
        self.files = files
    def write(self, s):
        return random.choice(self.files).write(s)
Sign up to request clarification or add additional context in comments.

2 Comments

I suspect that printing to the terminal in real-time is a red herring, and the OP really wants the redirected stdout to appear in his GUI in real-time.
@PM2Ring I couldn't figure out what the OP wants, so I wrote two different things to show how easy it is to write whatever he actually does want. Of course if he just copies and pastes either one without reading anything, the odds that it happens to do what he wants are a million to one, but I'm not too concerned with that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.