0

I've built a set of data pipes for moving data out of Elastic into RDS, using the Python Elasticsearch library to pass a SQL query to Elasticsearch. I'm trying to add a new one that will query all the APM indices.

When I run the SQL query in the ES devtools, I get the same results as in Kibana (~140000 documents). When I run the same query (copy-pasted! checked for weird character mutations!) in Python, I get ~3500 documents. Upon investigation, it seems that the Python script is only retrieving documents from indices that errors are routed to.

In Kibana, we group all our APM indices together, like traces-apm*,apm-*,logs-apm*,apm-*,metrics-apm*,apm-*. In Kibana Discovery/Elasticsearch devtools, queries against this grouping yield 140k docs. In Python against the same group I get 3500 docs.

Any idea what the difference could be? Versions of Elastic and versions of the python elasticsearch library maybe?? I'm at my wits end y'all, this doesn't make any sense.

Improve this question
Related questions
7
ElasticSearch: How to return only specific fields using ElasticSearch API?
4
Elasticsearch python API: Delete documents by query
634
Install a Python package into a different directory using pip?
Load 5 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.