Elasticsearch Service with Cognito -- how to access via Python HTTP request

Question

I recently setup an Elasticsearch service and configured it to use Cognito for identify management. I followed this guide and so far everything is working exactly as expected. I'm able to add new users as expected, and they can access Kibana as expected.

However, I would also like to use Python to interact with the Elasticsearch service. I've following this guide, but I get permission errors about not having the right access.

from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth
import boto3

host = 'hostname.us-east-2.es.amazonaws.com/'
region = 'us-east-2'
service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)

es = Elasticsearch(
    hosts = [{'host': host, 'port': 443}],
    http_auth = awsauth,
    use_ssl = True,
    verify_certs = True,
    connection_class = RequestsHttpConnection
)

print(es.info())

AuthorizationException: AuthorizationException(403, 'security_exception', 'no permissions for [indices:admin/get] and User [name=arn:aws:iam::12345678:user/username, backend_roles=[], requestedTenant=null]')

I'm not sure if the issue is related to the way that I have Conginto configured, or if it's related to the way that I'm submitting this request. Any help on figuring out where my issue might be would be greatly appreciated.

Can you disable/remove congito authetication and try without it. Also what policy to you have on your ES? — Marcin
– Marcin, Commented Jan 8, 2021 at 8:26

bengisu Pınar · Accepted Answer · 2021-01-12 22:36:38Z

2

It happened to me too. There are two ways I found, you have to create an IAM user for both of them.

Open IAM console and give necessary policies to IAM user for reaching your elasticsearch domain (or I attached "AdministratorAccess" policy only (It gives all access of AWS)), then you should use credentials of this IAM account.
Or open kibana -> security -> roles -> click all_access role -> mapped_users -> manage_mapping. Then you should add arn of your IAM user to backend roles. You have to use this user's credentials not "boto3.Session().get_credentials()" on your code to connect ES. I suggest the second one if you are not familiar with IAM roles and policies.

answered Jan 12, 2021 at 22:36

bengisu Pınar

462 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elasticsearch Service with Cognito -- how to access via Python HTTP request

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related