Paramiko: Asynchronous reading for long shell script output using 'select'

Question

I have following code from here with little modification:

#!/usr/bin/env python                                                  

import paramiko                                                        
import select                                                          


server = "192.168.100.100"                                                
port = 22                                                              
name = "root"                                                          
password = "pass"                                                    


def main():                                                            
    client = paramiko.SSHClient()                                      
    client.load_system_host_keys()                                     

    client.set_missing_host_key_policy(paramiko.MissingHostKeyPolicy())
    client.connect(server, port, name, password)                       

    channel = client.get_transport().open_session()                    
    channel.exec_command("/tmp/test.sh")                      

    while True:                                                        
        if channel.exit_status_ready():                                
            break                                                      

        r, w, x = select.select([channel], [], [], 0.0)                
        if len(r) > 0:                                                 
            print channel.recv(1024)                                   

if __name__ == "__main__":                                             
    main()

Where test.sh has following content:

#!/usr/bin/env bash
while true
do
    echo "Message"
    sleep 1
done

So after executing python script CPU usage by script goes up to 100%. It means this select function does not wait until one or more file descriptors are ready for some kind of I/O. As far as knew this is busy loop problem where 'while ... loop' will iterate continuously even data for read is not presented. How I can make it asynchronous read of remote output?

What do you want to happen while select is doing it's thing? You have code dependent on the output immediately after. — Eli
– Eli, Commented Jul 10, 2014 at 9:51
For instance if sleep time in bash script is equal to 5. Select.select should wait until output will be generated instead of looping without any sleep. Maybe I misunderstood of meaning 'select'. What I want to do is read generated output asynchronously. — Khamidulla
– Khamidulla, Commented Jul 10, 2014 at 9:54
You're using the word 'asynchronously' but based on your explanation, I don't think that's what you want. It sounds like you're saying you want select to block until it gets a result. You just don't want to have it loop endlessly, but instead wait without maxing out CPU. Is that right? — Eli
– Eli, Commented Jul 10, 2014 at 10:09

Eli · Accepted Answer · 2014-07-11 17:02:43Z

3

Your problem is you have timeout set to 0 seconds on select, so it doesn't block at all. The default is to block as long as necessary, so either take out the timeout parameter from select, or change it to something larger:

r, w, x = select.select([channel], [], [])

or

r, w, x = select.select([channel], [], [], 10.0)

You can see the difference both in CPU, and also by putting a simple print statement after your while true. In the 0.0 second timeout statement, you'll see it hit continuously. In the greater timeout case you'll see it hit exactly once (and cpu much lower).

answered Jul 11, 2014 at 17:02

Eli

39.3k40 gold badges156 silver badges213 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Paramiko: Asynchronous reading for long shell script output using 'select'

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related