transform Python dictionary with bytes to string in python 3.8

Question

I am hoping people can help me. I apologize in advance for any grammatical mistakes. I am working on large scale porting of LDAP data to MYSQL database. The LDAP dump from python program I "inherited". For performance and maintenance reasons, we are moving the data out of LDAP and into web authentication system that uses MySQL as the backend. For business reasons, the LDAP to webauth migration is going to happen over the next year -- so I need to keep the two systems in sync.

Having used python 2 for years --- Unicode / UTF-8 in python 3 is an area that causes me much pain.

The ldap dump produces the following output as part of the execution

{'sn': [b'Jones'], 'title': [b'WH Trainee'], 'givenName': [b'Example'], 'distinguishedName': [b'CN=Example Jones,OU=Warehouse Trainees,OU=GU,DC=jupiter,DC=somecorp,DC=org'], 'whenCreated': [b'20201027121144.0Z'], 'department': [b'WHSE'], 'sAMAccountName': [b'ejones'], 'manager': [b'CN=Bobby Smith,OU=WHSE,OU=GU,DC=jupiter,DC=somecorp,DC=org']}

Formatted for easier reading: ** existing output**

{
'sn': [b'Jones'], 
'title': [b'WH Trainee'], 
'givenName': [b'Example'], 
'distinguishedName': [b'CN=Example Jones,OU=Warehouse Trainees,OU=GU,DC=jupiter,DC=somecorp,DC=org'], 
'whenCreated': [b'20061027132244.0Z'], 
'department': [b'WHSE'], 
'sAMAccountName': [b'ejones'], 
'manager': [b'CN=Bobby Smith,OU=WHSE,OU=GU,DC=jupiter,DC=somecorp,DC=org']
}

We need to convert this output to simple string, key value pairs in a dictionary. I would like to have this implemented as a simple function that can be called aond convert all of the values in a dictionary at once. Unfortunately some ldap records are missing things like departments and managers, so the number of keys and values in the dict varies. I need a way to transform the byte values in the dict to plain ascii strings.

Desired transformation using single function call something like transformdict(dictname)

desired output

{
'sn': 'Jones', 
'title': 'WH Trainee', 
'givenName': 'Example', 
'distinguishedName': 'CN=Example Jones,OU=Warehouse Trainees,OU=GU,DC=jupiter,DC=somecorp,DC=org', 
'whenCreated': '20061027132244.0Z', 
'department': 'WHSE', 
'sAMAccountName': 'ejones', 
'manager': 'CN=Bobby Smith,OU=WHSE,OU=GU,DC=jupiter,DC=somecorp,DC=org'
}

We using python 3.8 environment.

Any suggestions?

Those are lists because some attributes may have multiple values. If something like [b"foo", b"bar"] is encountered, what should the resulting string be? Also, what encoding are these byte strings? utf-8, current code page? — tdelaney
– tdelaney, Commented Nov 18, 2020 at 4:58
Thank you for question. I checked our entire ldap extract. We do not have multi value attributes thankfully. LDAP standard is utf-8 codepage. — mrmatthew
– mrmatthew, Commented Nov 18, 2020 at 13:02

Serial Lazer · Accepted Answer · 2020-11-18 04:42:35Z

3

This should do it:

S = {'sn': [b'Jones'], 'title': [b'WH Trainee'], 'givenName': [b'Example'], 'distinguishedName': [b'CN=Example Jones,OU=Warehouse Trainees,OU=GU,DC=jupiter,DC=somecorp,DC=org'], 'whenCreated': [b'20201027121144.0Z'], 'department': [b'WHSE'], 'sAMAccountName': [b'ejones'], 'manager': [b'CN=Bobby Smith,OU=WHSE,OU=GU,DC=jupiter,DC=somecorp,DC=org']}

dt = {key: value[0].decode("utf-8") for key, value in S.items()}
print(dt)

Output:

{
    "sn": "Jones",
    "title": "WH Trainee",
    "givenName": "Example",
    "distinguishedName": "CN=Example Jones,OU=Warehouse Trainees,OU=GU,DC=jupiter,DC=somecorp,DC=org",
    "whenCreated": "20201027121144.0Z",
    "department": "WHSE",
    "sAMAccountName": "ejones",
    "manager": "CN=Bobby Smith,OU=WHSE,OU=GU,DC=jupiter,DC=somecorp,DC=org"
  }

answered Nov 18, 2020 at 4:42

Serial Lazer

1,6691 gold badge9 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mrmatthew Over a year ago

Thank you very much! I had tried a variant of this and had some serious syntax errors. I appreciate the assist.

Collectives™ on Stack Overflow

transform Python dictionary with bytes to string in python 3.8

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related