parsing specific strings from list items in python

Question

I have the following code in python, which contains log messages to debug SSH.

for log_item in ssh_log:
   print(log_item.rstrip())

#will show ...
2022-04-06 01:55:15,085 10.x Remote version/idstring: SSH-2.0-ConfD-4.3.11.4
2022-04-06 01:55:15,085 20.x Connected (version 2.0, client ConfD-4.3.11.4)
2022-04-06 01:55:15,161 10.x kex algos:['diffie-hellman-group14-sha1'] server key:['ssh-rsa']
...

What is the approach to get the values in bold assign my variables, maybe some regex as part of the for loop or something else to get the following:

idstring = SSH-2.0-ConfD-4.3.11.4
kex_algos = ['diffie-hellman-group14-sha1']
key_type = ['ssh-rsa']

regex might not be necessary for such a simple case. You might just use log_item.find() to filter out the relevant lines, then log_item.split() to split those into strings and pick the interesting ones by index. — Jussi Nurminen
– Jussi Nurminen, Commented Apr 6, 2022 at 6:26
Do all lines in the log file contain one of 'Remote", 'Connected' or 'kex'? The answers offered so far may work for this subset of log data but what if those tokens are not in the file? Also, why do you need kex_algos and key_type as lists? — jackal
– jackal, Commented Apr 6, 2022 at 7:02
Hi, yes all lines will have those elements. I don't actually need kex_algos or key_type as lists, its just happens their output in the log is like that — JON
– JON, Commented Apr 6, 2022 at 17:24

Manjunath K Mayya · Accepted Answer · 2022-04-06 06:57:27Z

1

Solution without regex. See comments inline below.

for log_item in ssh_log:
    line = log_item.rstrip()
    if 'idstring' in line:
        print('idstring = ',line.split(':')[-1]) #Pick last value after ':'
    if 'kex algos' in line:
        print('kex algos = ', line[line.find('['):line.find(']')+1]) #find between first set of square brackets.
    if 'key:' in line:
        key = line.split('key:')[1] #Get values after 'key:'
        print('key_type = ', key)

You can update prints to variable assignments if this is what you required.

answered Apr 6, 2022 at 6:57

Manjunath K Mayya

1,1161 gold badge11 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Anshumaan Mishra · Accepted Answer · 2022-04-06 06:34:06Z

If all the data is in the same format as the data given here, You can use the following regex:

import re
a = """
2022-04-06 01:55:15,085 10.x Remote version/idstring: SSH-2.0-ConfD-4.3.11.4
2022-04-06 01:55:15,085 20.x Connected (version 2.0, client ConfD-4.3.11.4)
2022-04-06 01:55:15,161 10.x kex algos:['diffie-hellman-group14-sha1'] server key:['ssh-rsa']"""

idstring = re.findall("idstring: (.*)", a)[0] # Remove zero to get a list if 
                                              # multiple values are present
print(idstring)
kex_algos = re.findall("algos:\['(.*)'\] ", a)
print(kex_algos)
key_type = re.findall("key:\['(.*)'\]", a)
print(key_type)

Output:

'SSH-2.0-ConfD-4.3.11.4'
['diffie-hellman-group14-sha1']
['ssh-rsa']

Baris Ozensel · Accepted Answer · 2022-04-06 07:03:21Z

You can also use ttp template to parse your data if your data has similar structure.

from ttp import ttp
import json

with open("log.txt") as f:
    data_to_parse = f.read()

ttp_template = """
{{ignore}} {{ignore}} {{ignore}} {{ignore}} version/idstring: {{version_id_string}}
{{ignore}} {{ignore}} {{ignore}} {{ignore}} algos:{{key_algos}} server key:{{key_type}}
"""

parser = ttp(data=data_to_parse, template=ttp_template)
parser.parse()

# print result in JSON format
results = parser.result(format='json')[0]
# print(results)

result = json.loads(results)

# print(result)

for i in result:
    print(i["key_algos"])
    print(i["key_type"])
    print(i["version_id_string"])

The output is :

['diffie-hellman-group14-sha1']
['ssh-rsa']
SSH-2.0-ConfD-4.3.11.4

jackal · Accepted Answer · 2022-04-06 07:34:52Z

With the 3 lines of sample data from the original question in a file, one could take this approach:

import re

with open('ssh.log') as sshlog:
    for line in map(str.strip, sshlog):
        _, _, _, kw, *rem = line.split()
        match kw:
            case 'Remote':
                print(f'ID string = {rem[-1]}')
            case 'kex':
                m = re.findall("(?<=\[').+?(?='\])", line)
                print(f'algos = {m[0]}')
                print(f'type = {m[1]}')
            case _:
                pass

The assumption here is that only lines with either of the keywords 'Remote' or 'kex' are of interest.

Output:

ID string = SSH-2.0-ConfD-4.3.11.4
algos = diffie-hellman-group14-sha1
type = ssh-rsa

Collectives™ on Stack Overflow

parsing specific strings from list items in python

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related