Elastic search to sort with timestamp using python

Question

I have created a python api which will accept the *.json file.

for ex: abc.json

{
    "service_name": "httpd",
    "service_status": "DOWN",
    "host_name": "host1",
    "time": "1616600149.014236"
}

and, push the data to the ES using below python api.

@app.route('/add', methods=['POST']) def insert_data():
    #directory = '/home/user'
    dir = os.getcwd()
    os.chdir(dir)
    i = 1
    #This function will read all the json payload in the given dir and upload to ES.
    for filename in os.listdir(dir):
        if filename.endswith(".json"):
            f = open(filename)
            status_content = f.read()
            # Send the data into es
            result=(es.index(index='svc_index', ignore=400, doc_type='doc',
            id=i, body=json.loads(status_content)))
            i = i + 1
            print("result")

    return jsonify(result)

Output:

{
        "_id": "4", 
        "_index": "svc_index", 
        "_score": 1.0, 
        "_source": {
          "host_name": "host1", 
          "service_name": "httpd", 
          "service_status": "DOWN", 
          "time": "1616600143.5427265"
        }, 
        "_type": "doc"
      },

Since timestamp is being stored as string, sorting is not working. I wanted to bring the latest result on top. Could anyone please help into this.

Thanks!!

stackoverflow.com/questions/50130208/…

Alex W
– Alex W

2021-03-24 17:37:53 +00:00
Commented Mar 24, 2021 at 17:37 — Alex W
– Alex W, Commented Mar 24, 2021 at 17:37

harper · Accepted Answer · 2021-03-24 22:48:48Z

2

I have found the solution.

timestamp is getting sorted in desc order with the following query in python.

result = es.search(index="svc_index", doc_type="doc", body={"query": {"match": {"service_name": query}},"sort":{"time": {'order': 'desc', 'mode':'max'}}}, size=1)

answered Mar 24, 2021 at 22:48

harper

838 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

illogicalhawk · Accepted Answer · 2021-03-24 17:32:59Z

1

If you want Elastic to know that the Time property is a timestamp and treat it as such, you can specify it as a Date field in the index mapping:

https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html

That should enable Elastic to sort it correctly.

answered Mar 24, 2021 at 17:32

illogicalhawk

111 bronze badge

1 Comment

harper Over a year ago

Thank you, I have used mode under sort option.Its working.

Collectives™ on Stack Overflow

Elastic search to sort with timestamp using python

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related