PyMongo: querying using json with a list

Question

If I have some json like this:

query = {
   "canonical.operatingSystemFamily": "Linux",
   "canonical.region": "us-east-1"
}

I can pass that directly into find and it works:

self._db_conn[collection_name].find(query)

I want to do the same, but with lists, e.g.:

query = {
    "canonical.operatingSystemFamily": ["Linux","Windows"],
    "canonical.region": ["us-east-1", "us-east-2"]
}

Is there a way to do that? I know about "$in" but I do not want to have to parse my query data. This is a very simplified example, and there are a lot of fields that may or may not be there. Is there a way to use that json with lists directly?

You know about $in which is good. You don't want to parse query data -- but how did you get ["Linux","Windows"] attached to canonical.operatingSystemFamily? That didn't come directly from the app did it? — Buzz Moschetti
– Buzz Moschetti, Commented Mar 2, 2022 at 0:24
JSON data types include lists, objects, strings, numbers and null. Also, you can always verify your query string in any JSON formatter / validator. — prasad_
– prasad_, Commented Mar 2, 2022 at 3:18
@BuzzMoschetti it is part of a much larger data structure that comes from an external source. What I showed is a simplified example to show my question. In the app the entire data structure is passed to find and it works except when there is a list in the data. — Larry Martell
– Larry Martell, Commented Mar 2, 2022 at 13:36
@prasad_ I know all about JSON. How does your reply help answer my question? — Larry Martell
– Larry Martell, Commented Mar 2, 2022 at 13:37
What are the types of the fields "canonical.operatingSystemFamily" and "canonical.region" ? — prasad_
– prasad_, Commented Mar 2, 2022 at 14:30

rickhg12hs · Accepted Answer · 2022-03-02 16:17:10Z

1

It's not much of a parser - just checking type.

$ ipython
Python 3.8.10 (default, May  4 2021, 00:00:00) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.1.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: query = {
   ...:     "canonical.operatingSystemFamily": ["Linux","Windows"],
   ...:     "canonical.region": ["us-east-1", "us-east-2"]
   ...: }

In [2]: for k,v in query.items():
   ...:     if type(v)==list:
   ...:         query[k]={"$in": v}
   ...: 

In [3]: query
Out[3]: 
{'canonical.operatingSystemFamily': {'$in': ['Linux', 'Windows']},
 'canonical.region': {'$in': ['us-east-1', 'us-east-2']}}

If you wanted, you could define a function to transform query when a list is present.

from copy import deepcopy
def query_lfixer(query, copy=True):
    if copy:
        query = deepcopy(query)
    for k, v in query.items():
        if type(v)==list:
            query[k] = {"$in": v}
    return query

And then you could use it like:

self._db_conn[collection_name].find(query_lfixer(query))

Or you could transform query in any number of other ways too.

edited Mar 2, 2022 at 16:17

answered Mar 2, 2022 at 5:44

rickhg12hs

12k6 gold badges31 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Larry Martell Over a year ago

This is part of a much larger data structure that comes from an external source. What I showed is a simplified example to show my question. In the app the entire data structure is passed to find and it works except when there is a list in the data.

Buzz Moschetti Over a year ago

@LarryMartell This will work on an arbitrarily large input query and convert any list to an $in expression. Should do the trick.

rickhg12hs Over a year ago

@LarryMartell All this does is modify the query you showed so that list's would use {"$in" .... All other field types are untouched.

Larry Martell Over a year ago

Currently the code does this: self._db_conn[collection_name].find(query) If I use the code above how would I pass in both the query and the $in clause?

rickhg12hs Over a year ago

@LarryMartell There are lots of possibilities. I updated my answer with one way someone could do it.

|

Collectives™ on Stack Overflow

PyMongo: querying using json with a list

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related