Is there an expression-based tool for querying python complex objects the same way one can do using XPath for XML or jsonpath for JSON?
I thought about serializing my object to JSON and then using jsonpath on it, but it seems to be a clumsy way of doing that.
You can use built-in library json to import json as a nested dictionary and traverse it using dictionary notation - root['level1_object']['level2_object']. JSON-compatible object types are of course loaded as corresponding Python types.
For other types of data there are other libraries, which mostly behave in similar fashion.
My new favourite is Box, which allows you to traverse nested dictionaries using a dot notation.
You might want to take a look at AST module: https://docs.python.org/2/library/ast.html
@vBobCat I'm currently on the search for a similar solution. Agreed that serializing and deserializing with json is not ideal. What did you end up going with?
I found http://objectpath.org/ to be close to the right solution for my use case though it lacks features in making arbitrary updates to fields that I need. Its syntax, though slightly different than JSONPath, expresses many of the things that JSONPath does.
Box library (in accepted answer) combined with exec(), so I can interpolate stored paths in a string and execute it.
I'm adding this answer for the sake of future researchers:
It seems jsonpath-rw is the library I was looking for since the beginning, since it does exactly what I originally requested.
Adding a non-library option here. A method that finds a nested element based on a dot notation string (can traverse nested dicts and lists), see below of Gist here:
from functools import reduce
import re
from typing import Any, Optional
def find_key(dot_notation_path: str, payload: dict) -> Any:
"""Try to get a deep value from a dict based on a dot-notation"""
def get_despite_none(payload: Optional[dict], key: str) -> Any:
"""Try to get value from dict, even if dict is None"""
if not payload or not isinstance(payload, (dict, list)):
return None
# can also access lists if needed, e.g., if key is '[1]'
if (num_key := re.match(r"^\[(\d+)\]$", key)) is not None:
try:
return payload[int(num_key.group(1))]
except IndexError:
return None
else:
return payload.get(key, None)
found = reduce(get_despite_none, dot_notation_path.split("."), payload)
# compare to None, as the key could exist and be empty
if found is None:
raise KeyError()
return found
# Test cases:
payload = {
"haystack1": {
"haystack2": {
"haystack3": None,
"haystack4": "needle"
}
},
"haystack5": [
{"haystack6": None},
{"haystack7": "needle"}
],
"haystack8": {},
}
find_key("haystack1.haystack2.haystack4", payload)
# "needle"
find_key("haystack5.[1].haystack7", payload)
# "needle"
find_key("[0].haystack5.[1].haystack7", [payload, None])
# "needle"
find_key("haystack8", payload)
# {}
find_key("haystack1.haystack2.haystack4.haystack99", payload)
# KeyError
