3

I've got some code that runs through a list of devices, connects to them via SSH and queries some parameters. The basic code looks like this:

ssh = paramiko.SSHClient()
ssh.connect(ip_address, username='root', password=password)

try:
    ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command('uname -r')
    if ssh_stdout.channel.recv_exit_status() == 0:
        temp = ssh_stdout.readlines()
        if temp:
            kernel_version = temp[0].strip('\n').strip('\"')
except BaseException as ex:
    print(f'An exception occurred: {ex}')

try:
    ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command('lsb_release -rs')
    if ssh_stdout.channel.recv_exit_status() == 0:
        temp = ssh_stdout.readlines()
        if temp:
            ubuntu_version = temp[0].strip('\n').strip('\"')
except BaseException as ex:
    print(f'An exception occurred: {ex}')

But there are a lot of devices and it takes a long time to run. It's faster to run multiple commands in one go:

try:
    ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command('uname -r; lsb_release -rs')
    if ssh_stdout.channel.recv_exit_status() == 0:
        temp = ssh_stdout.readlines()
        if temp[0]:
            kernel_version = temp[0].strip('\n').strip('\"')
        if temp[1]:
            ubuntu_version = temp[1].strip('\n').strip('\"')
except BaseException as ex:
    print(f'An exception occurred: {ex}')

but how can I handle one command passing and one failing? Is there a better way of doing this?

1
  • 1
    You can improve your second code by printing an exit code of each command and parsing it from the output. Or you can parallelize connections to multiple servers to speed up the overal operation. Commented Jun 8 at 18:08

1 Answer 1

2

You can implement parallelism on top of this by using multithreading to reduce time.

import paramiko


def get_kernel_ubuntu_version_raw(ip_address, password):
    ssh = paramiko.SSHClient()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh.connect(ip_address, username="root", password=password)

    try:
        ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("uname -r")
        if ssh_stdout.channel.recv_exit_status() == 0:
            temp = ssh_stdout.readlines()
            if temp:
                kernel_version = temp[0].strip("\n").strip('"')
    except BaseException as ex:
        print(f"An exception occurred: {ex}")
    try:
        ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("lsb_release -rs")
        if ssh_stdout.channel.recv_exit_status() == 0:
            temp = ssh_stdout.readlines()
            if temp:
                ubuntu_version = temp[0].strip("\n").strip('"')
    except BaseException as ex:
        print(f"An exception occurred: {ex}")

And I try to use send&recv instead of exec_command.


def get_kernel_ubuntu_version(ip_address, password):
    prompt = r"#\s+$"
    ssh = paramiko.SSHClient()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh.connect(ip_address, username="root", password=password)
    chan = ssh.get_transport().open_session()
    chan.get_pty()
    chan.invoke_shell()

    try:
        chan.send(b"uname -r\n")
        kernel_version = ""
        while not re.search(prompt, kernel_version):
            kernel_version += chan.recv(1024).decode(errors="ignore")
        print(f"{ip_address}-kernel_version: {kernel_version}")
    except BaseException as ex:
        print(f"An exception occurred: {ex}")

    try:
        chan.send(b"lsb_release -rs\n")
        ubuntu_version = ""
        while not re.search(prompt, ubuntu_version):
            ubuntu_version += chan.recv(1024).decode(errors="ignore")
        print(f"{ip_address}-kernel_version: {ubuntu_version}")
    except BaseException as ex:
        print(f"An exception occurred: {ex}")

Test code


from concurrent.futures import ThreadPoolExecutor, wait
import time

if __name__ == "__main__":
    tests = [
    ]

    start = time.time()
    for ip, pwd in tests:
        get_kernel_ubuntu_version_raw(ip, pwd)
    end = time.time()
    t1 = end - start

    start = time.time()
    for ip, pwd in tests:
        get_kernel_ubuntu_version(ip, pwd)
    end = time.time()
    t2 = end - start

    start = time.time()
    with ThreadPoolExecutor(max_workers=8) as pool:
        all_task = []
        for ip, pwd in tests:
            task = pool.submit(get_kernel_ubuntu_version_raw, ip, pwd)
            all_task.append(task)
        wait(all_task)
    end = time.time()
    t3 = end - start
    
    start = time.time()
    with ThreadPoolExecutor(max_workers=8) as pool:
        all_task = []
        for ip, pwd in tests:
            task = pool.submit(get_kernel_ubuntu_version, ip, pwd)
            all_task.append(task)
        wait(all_task)
    end = time.time()
    t4 = end - start
    print(t1)
    print(t2)
    print(t3)
    print(t4)

I tried 30 devices.

The original code took 32.69 seconds

send&revc code took 28.23 seconds

ThreadPool code took 6.15 seconds

send&revc&ThreadPool` code took 5.73 seconds

Sign up to request clarification or add additional context in comments.

6 Comments

I tried the first example and "uname -r" works but then "lsb_release -rs" throws an exception Channel is not open. Seems like you can only execute one command per channel? stackoverflow.com/a/28883457/264822
This code works well on my pc. chan.exec_command will not close channel, while ssh.exec_command dose.
Not what the documentation says docs.paramiko.org/en/stable/api/…
Yes you are right. I have changed my answer. And I try to use send&recv instead of exec_command. Time reduced by about 4 seconds from 32.69 to 28.23, but this can be ignored in multi-threading.
In the first example where's the multithreading? All youve done is added ssh.set_missing_host_key_policy()
multithreading is in Test code ThreadPoolExecutor

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.