Unable to access ElasticSearch AWS through Python

Question

I'm trying to access ElasticSearch AWS from my localhost through Python (I can access it through my browser).

from elasticsearch import Elasticsearch
ELASTIC_SEARCH_ENDPOINT = 'https://xxx'
es = Elasticsearch([ELASTIC_SEARCH_ENDPOINT])

I'm receiving this error:

ImproperlyConfigured('Root certificates are missing for certificate validation. Either pass them in using the ca_certs parameter or install certifi to use it automatically.',)

How can I access it? I have not configured any certificate, I only liberated the IPs that can access ElasticSearch Service.

Bhardwajhack · Accepted Answer · 2017-12-12 17:11:52Z

21

for python 3.5 install certifi and use ca_certs=certifi.where() this will pass the certificates

import certifi
from elasticsearch import Elasticsearch

host = 'https://###########.ap-south-1.es.amazonaws.com'

es = Elasticsearch([host], use_ssl=True, ca_certs=certifi.where())

answered Dec 12, 2017 at 17:11

Bhardwajhack

2362 silver badges4 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ajay Singh · Accepted Answer · 2017-03-28 10:09:19Z

elasticsearch-py doesn’t ship with default set of root certificates. To have working SSL certificate validation you need to either specify your own as ca_certs or install certifi which will be picked up automatically.

from elasticsearch import Elasticsearch

# you can use RFC-1738 to specify the url
es = Elasticsearch(['https://user:secret@localhost:443'])

# ... or specify common parameters as kwargs

# use certifi for CA certificates
import certifi

es = Elasticsearch(
    ['localhost', 'otherhost'],
    http_auth=('user', 'secret'),
    port=443,
    use_ssl=True 
)

# SSL client authentication using client_cert and client_key

es = Elasticsearch(
    ['localhost', 'otherhost'],
    http_auth=('user', 'secret'),
    port=443,
    use_ssl=True,
    ca_certs='/path/to/cacert.pem',
    client_cert='/path/to/client_cert.pem',
    client_key='/path/to/client_key.pem',
)

https://elasticsearch-py.readthedocs.io/en/master/

Edhowler · Accepted Answer · 2017-05-30 22:25:17Z

6

I did it this way and it worked:

from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth

host = 'YOURHOST.us-east-1.es.amazonaws.com'
awsauth = AWS4Auth(YOUR_ACCESS_KEY, YOUR_SECRET_KEY, REGION, 'es')

es = Elasticsearch(
    hosts=[{'host': host, 'port': 443}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
)
print(es.info())

answered May 30, 2017 at 22:25

Edhowler

83510 silver badges18 bronze badges

Comments

kartik · Accepted Answer · 2018-04-19 10:00:24Z

2

You can also use boto3 to generate tempory access key & secret key.

import boto3

region = 'ap-southeast-2'
service = 'es'
session = boto3.Session()
credentials = session.get_credentials()

awsauth = AWS4Auth(credentials.access_key, credentials.secret_key,region, service,session_token=credentials.token)

es = Elasticsearch(
    hosts = [{'host': host, 'port': 443}],
    http_auth = awsauth,
    use_ssl = True,
    verify_certs = True,
    connection_class = RequestsHttpConnection
)

https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-indexing.html

edited Apr 19, 2018 at 10:00

answered Apr 19, 2018 at 9:55

kartik

2,1353 gold badges21 silver badges31 bronze badges

Collectives™ on Stack Overflow

Unable to access ElasticSearch AWS through Python

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related