Querying ElasticSearch with Python Requests not working fine

Question

I'm trying to do full-text search on a mongodb db with the Elastic Search engine but I ran into a problem: no matters what search term I provide(or if I use query1 or query2), the engine always returns the same results. I think the problem is in the way I make the requests, but I don't know how to solve it.

Here is the code:

def search(search_term):
    query1 = {
        "fuzzy" : {
            "art_text" : {
                "value" : search_term,
                "boost" : 1.0,
                "min_similarity" : 0.5,
                "prefix_length" : 0
            }
        },
        "filter": {
            "range" : {
                "published": {
                    "from" : "20130409T000000",
                    "to": "20130410T235959"
                }
            }
        }
    }
    query2 = {
        "match_phrase": { "art_text": search_term }
    }

    es_query = json.dumps(query1)
    uri = 'http://localhost:9200/newsidx/_search'
    r = requests.get(uri, params=es_query)
    results = json.loads( r.text )
    data = [res['_source']['api_id'] for res in results['hits']['hits'] ]
    print "results: %d" % len(data)
    pprint(data)

Just eyeballing this quickly... does your "fuzzy" clause need to be wrapped inside a "query"? So the structure would become "query": { "fuzzy": { ... } }, "filter": {...}. Can you post the result you are getting from your request so we can try and see if there is anything obvious. — Phil
– Phil, Commented Apr 11, 2013 at 13:37
The request returns 10 items, no matter what search_term I provide.I dont know what I'm doing wrong. — Rod0n
– Rod0n, Commented Apr 11, 2013 at 19:01
Is the data returned real? Are the documents valid documents, or something else? Have you tried a simple uri search that specifies the document type and search param all in one? For example: curl -XGET 'localhost:9200/newsidx/some_type/_search?q=art_text:hello' . Or maybe something that searches a tag that isn't your art_text field. Just trying to help you narrow things down — Phil
– Phil, Commented Apr 11, 2013 at 22:39

Ian Stapleton Cordasco · Accepted Answer · 2013-04-12 23:44:46Z

20

The params parameter is not for data being sent. If you're trying to send data to the server you should specifically be using the data parameter. If you're trying to send query parameters, then you shouldn't be JSON-encoding them and just give it to params as a dict.

I suspect your first request should be the following:

r = requests.get(uri, data=es_query)

And before someone downvotes me, yes the HTTP/1.1 spec allows data to be sent with GET requests and yes requests does support it.

answered Apr 12, 2013 at 23:44

Ian Stapleton Cordasco

29.2k4 gold badges73 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

justdan23 Over a year ago

"get" works as it passes 'data=' as a url parameter in the HTTP request. If you need larger sets of data, "post" will work too, but should be saved for writing data to documents. Another benefit to "post" is it makes it harder to reproduce queries in the browser when people are probing traffic.

Cleb · Accepted Answer · 2016-01-08 14:57:53Z

-2

search = {'query': {'match': {'test_id':13} }, 'sort' {'date_utc':{'order':'desc'}} }

data = requests.get('http://localhost:9200/newsidx/test/_search?&pretty',params = search)
print data.json()

http://docs.python-requests.org/en/latest/user/quickstart/

edited Jan 8, 2016 at 14:57

Cleb

26.2k23 gold badges129 silver badges164 bronze badges

answered Jan 8, 2016 at 14:01

alfonsoolavarria

1,22512 silver badges11 bronze badges

Collectives™ on Stack Overflow

Querying ElasticSearch with Python Requests not working fine

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest