Text search in aggregation using pymongo

Question

I have a collection named users, it has following attributes

{
“_id”: “937a04d3f516443e87abe8308a1fe83e”,
“username”: “andy”,
“full_name”: “andy white”,
“image” : “https://example.com/xyz.jpg”,
… etc
}

i want to make a text search on full_name and username using aggregation pipeline, so that if a user search for any 3 letters, then the most relevant full_name or username returned sorted by relevancy, i have already created text index on username and full_name and then i tried query from below link:

https://www.mongodb.com/docs/manual/tutorial/text-search-in-aggregation/#return-results-sorted-by-text-search-score

pipeline_stage = [
{"$match": {"$text": {"$search": “whit”}}},
{"$sort": {“score”: {"$meta": “textScore”}}},
{"$project": {“username”: 1,“full_name”: 1,“image”:1}}
]

stages = [*pipeline_stage]
users = users_db.aggregate(stages)

but i am getting below error:

pymongo.errors.OperationFailure: FieldPath field names may not start with ‘$’. Consider using $getField or $setField., full error: {‘ok’: 0.0, ‘errmsg’: “FieldPath field names may not start with ‘$’. Consider using $getField or $setField.”, ‘code’: 16410, ‘codeName’: ‘Location16410’, ‘$clusterTime’: {‘clusterTime’: Timestamp(1657811022, 14), ‘signature’: {‘hash’: b’a\xb4rem\x02\xc3\xa2P\x93E\nS\x1e\xa6\xaa\xb0\xb1\x85\xb5’, ‘keyId’: 7062773414158663703}}, ‘operationTime’: Timestamp(1657811022, 14)}

I also tried below link (my query also below) but i am getting full text search results, not working for partial text search: https://www.mongodb.com/docs/manual/tutorial/text-search-in-aggregation/#match-on-text-score

pipeline_stage = [
    {"$match": {"$text": {"$search": search_key}}},
    {"$project": {"full_name": 1, "score": {"$meta": "textScore"}}},
]

Any help will be appreciated,

Note: I want to do partial text search, sorted by relevant records at top,

Thanks

This issue get resolved if i remove $project from the query, but with $project i am unable to run the query, with the error i mentioned above — Xeeshan007
– Xeeshan007, Commented Jul 15, 2022 at 7:48
"$search" is "A string of terms". I don't think you can use it to do a "partial text search". Perhaps a regex could be used. — rickhg12hs
– rickhg12hs, Commented Jul 15, 2022 at 14:05
you are right @rickhg12hs, i am unable to do partial text search using $search in pipeline, do you have any other way to do a partial search and sort the relevant results on top in response — Xeeshan007
– Xeeshan007, Commented Jul 15, 2022 at 15:11
I would explore regexes and/or a "$function" that calculates some sort of "relevancy" or similarity (levenshtein distance?). — rickhg12hs
– rickhg12hs, Commented Jul 15, 2022 at 21:27

Belly Buster · Accepted Answer · 2022-07-15 20:47:56Z

0

Your project stage is incorrect, it should be

pipeline_stage = [
    {"$match": {"$text": {"$search": "and"}}},
    {"$sort": {"score": {"$meta": "textScore"}}},
    {"$project": { "username": "$username", "full_name": "$full_name", "image": "$image"}}
]

Also note if you use an English text search, words like and are not indexed.

answered Jul 15, 2022 at 20:47

Belly Buster

8,9142 gold badges12 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Xeeshan007 Over a year ago

Thanks for your answer, but after writing project stage i am still getting the same error i.e: pymongo.errors.OperationFailure: FieldPath field names may not start with '$'. Consider using $getField or $setField., full error: {'ok': 0.0, 'errmsg': "FieldPath field names may not start with '$'. Consider using $getField or $setField.", 'code': 16410, 'codeName': 'Location16410'...

Xeeshan007 Over a year ago

and here "and" means initials of any name like andrew, andy to be searched, not conjunction literally

Belly Buster Over a year ago

re point 1 what version of mongodb any pymongo are you running. Re point 2 and is on the stop list of english words c.f. github.com/mongodb/mongo/blob/master/src/mongo/db/fts/…

Xeeshan007 Over a year ago

i am using MongoDB 5.0.9 Enterprise, Python 3.9.7 and pymongo 4.1.1, and i understand about the and, so leave it, its not my concern, my concern is partial search with relevant words on top, thats it

Belly Buster Over a year ago

Have you tried replacing and with andy?

|

Collectives™ on Stack Overflow

Text search in aggregation using pymongo

1 Answer 1

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related