Import PostgreSQL with Sqoop in Docker

Question

I have a PostgreSQL DB sitting on my local machine (Windows) and I would like to import it into my Hortonworks Sandbox using Apache Sqoop. While something like this sounds great, the complicating factor is that my Sandbox is sitting in a Docker container, so statements such as sqoop list-tables --connect jdbc:postgresql://127.0.0.1/ambari --username ambari -P seem to run into authentication errors. I believe the issue comes from trying to connect to the local host from inside the docker container.

I looked at this post on connecting to a MySQL DB from within a container and this one to try to use PostgreSQL instead, but have so far been unsuccessful. I have tried connecting to '127.0.0.1' and '172.17.0.1' (the host's IP) in order to connect to my local host from within Docker. I have also adjusted PostgreSQL's configuration file to listen for connections on all IP addresses. However, I still get the following error messages when I run sqoop list-tables --connect jdbc:postgresql://<ip>:5432/<db_name> --username postgres -P (where <ip> is either 127.0.0.1 or 172.17.0.1, and <db_name> is the name of my database)

For connecting with 127.0.0.1:

ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: org.postgresql.util.PSQLException: FATAL: Ident authentication failed for user "postgres"

For connecting with 172.17.0.1:

Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

Any suggestions would be very helpful!

Alan Kavanagh · Accepted Answer · 2017-10-17 16:34:10Z

0

If this is just for local testing and not for production level coding, you can enable all trusted connections to your database by updating the pg_hba.conf file

Locate your pg_hba.conf file inside your postgres data directory
Vim the file and update it with the following lines:

#TYPE DATABASE USER ADDRESS METHOD local all all trust host all all 0.0.0.0/0 trust host all all ::1/128 trust
Restart your postgres service

If you do this, your first use case (using 127.0.0.1) should work

answered Oct 17, 2017 at 16:34

Alan Kavanagh

9,9338 gold badges46 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

tgordon18 Over a year ago

Sorry I should've mentioned that I'm doing this on Windows, which means the local/all/all/trust doesn't work (see this article)

Collectives™ on Stack Overflow

Import PostgreSQL with Sqoop in Docker

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related