Fetch value of a specific key from nested dictionary in pandas with python

Question

I am iterating through the rows in pandas dataframe printing out nested dictionaries from the specific column. My nested dictionary looks like this:

{'dek': "<p>Don't forget to buy a card</p>",
 'links': {'edit': {'dev': '//patty-menshealth.feature.net/en/content/edit/76517422-96ad-4b5c-a24a-c080c58bce0c',
   'prod': '//patty-menshealth.prod.com/en/content/edit/76517422-96ad-4b5c-a24a-c080c58bce0c',
   'stage': '//patty-menshealth.stage.net/en/content/edit/76517422-96ad-4b5c-a24a-c080c58bce0c'},
  'frontend': {'dev': '//menshealth.feature.net/trending-news/a19521193/fathers-day-weekend-plans/',
   'prod': '//www.menshealth.com/trending-news/a19521193/fathers-day-weekend-plans/',
   'stage': '//menshealth.stage.net/trending-news/a19521193/fathers-day-weekend-plans/'}},
 'header': {'title_color': 1, 'title_layout': 1},
 'sponsor': {'program_type': 1, 'tracking_urls': []},
 'social_dek': "<p>Don't forget to buy a card</p>",
 'auto_social': 0,
 'index_title': "\u200bWeekend Guide: Treat Your Dad Right This Father's Day",
 'short_title': "Treat Your Dad Right This Father's Day",
 'social_title': "\u200bWeekend Guide: Treat Your Dad Right This Father's Day",
 'editors_notes': '<p>nid: 2801076<br>created_date: 2017-06-16 13:00:01<br>compass_feed_date: 2017-06-21 14:01:58<br>contract_id: 40</p>',
 'seo_meta_title': "Treat Your Dad Right This Father's Day\u200b | Men’s Health",
 'social_share_url': '/trending-news/a19521193/fathers-day-weekend-plans/',
 'seo_related_links': {},
 'editor_attribution': 'by',
 'hide_from_homepage': 1,
 'syndication_rights': 3,
 'seo_meta_description': "\u200bFrom gifts to food ideas, we've got your Father's Day covered. Just don't forget to buy him a card."}

I use the code below:

def recursive_items(dictionary):
    for key, value in dictionary.iteritems():
        if type(value) is dict:
            yield from recursive_items(value)
        else:
            yield (key, value)

for key, value in recursive_items(merged_df["metadata_y"]):
    print(key, value)

How do I grab the value of a specific key? I tried to include the index of the key I am looking to fetch with print(key[5], value it gave me an error: TypeError: 'int' object is not subscriptable.

How can I grab the value?

Peter Leimbigler · Accepted Answer · 2020-02-07 21:08:13Z

Apologies about not directly addressing the original question, but maybe it's worth "flattening" the nested column using json_normalize.

For example, if your example data is named dictionary:

from pandas.io.json import json_normalize

# Flatten the nested dict, resulting in a DataFrame with 1 row and 23 columns
this_df = json_normalize(dictionary)

# Inspect the resulting columns. Is this structure useful?
this_df.columns
Index(['dek', 'social_dek', 'auto_social', 'index_title', 'short_title',
       'social_title', 'editors_notes', 'seo_meta_title', 'social_share_url',
       'editor_attribution', 'hide_from_homepage', 'syndication_rights',
       'seo_meta_description', 'links.edit.dev', 'links.edit.prod',
       'links.edit.stage', 'links.frontend.dev', 'links.frontend.prod',
       'links.frontend.stage', 'header.title_color', 'header.title_layout',
       'sponsor.program_type', 'sponsor.tracking_urls'],
      dtype='object')

koalaok · Accepted Answer · 2020-02-07 21:49:28Z

data={'dek': "<p>Don't forget to buy a card</p>",
'links': {'edit': {'dev': '//patty-menshealth.feature.net/en/content/edit/76517422-96ad-4b5c-a24a-c080c58bce0c',
'prod': '//patty-menshealth.prod.com/en/content/edit/76517422-96ad-4b5c-a24a-c080c58bce0c',
'stage': '//patty-menshealth.stage.net/en/content/edit/76517422-96ad-4b5c-a24a-c080c58bce0c'},
'frontend': {'dev': '//menshealth.feature.net/trending-news/a19521193/fathers-day-weekend-plans/',
'prod': '//www.menshealth.com/trending-news/a19521193/fathers-day-weekend-plans/',
'stage': '//menshealth.stage.net/trending-news/a19521193/fathers-day-weekend-plans/'}},
'header': {'title_color': 1, 'title_layout': 1},
'sponsor': {'program_type': 1, 'tracking_urls': []},
'social_dek': "<p>Don't forget to buy a card</p>",
'auto_social': 0,
'index_title': "\u200bWeekend Guide: Treat Your Dad Right This Father's Day",
'short_title': "Treat Your Dad Right This Father's Day",
'social_title': "\u200bWeekend Guide: Treat Your Dad Right This Father's Day",
'editors_notes': '<p>nid: 2801076<br>created_date: 2017-06-16 13:00:01<br>compass_feed_date: 2017-06-21 14:01:58<br>contract_id: 40</p>',
'seo_meta_title': "Treat Your Dad Right This Father's Day\u200b | Men’s Health",
'social_share_url': '/trending-news/a19521193/fathers-day-weekend-plans/',
'seo_related_links': {},
'editor_attribution': 'by',
'hide_from_homepage': 1,
'syndication_rights': 3,
'seo_meta_description': "\u200bFrom gifts to food ideas, we've got your Father's Day covered. Just don't forget to buy him a card."}

def findValByKey(key:str, data:dict, level=0):
    for i in data.keys():
        print("{} {}".format(level*'-', i))
        if i==key:
            print('---found---')
            return data[key]
        elif isinstance(data[i], dict):
            res = findValByKey(key, data[i], level+1)
            if res:
                return res


print(findValByKey('title_color', data))

Collectives™ on Stack Overflow

Fetch value of a specific key from nested dictionary in pandas with python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related