Copy data from S3 file into aws postgresql: invalid argument?

Question

I am trying to copy the content of an AWS S3 file into a AWS RDS Postgresql table but seem to get my syntax wrong?

COPY myschema.mytable FROM 'S3://bucket-name/file_name.csv' (FORMAT CSV, DELIMITER E'\t', ENCODING 'utf-8');

returns the following error: ERROR: could not open file "S3://bucket-name/file_name.csv" for reading: Invalid argument

Any idea what I am doing wrong?

Sébastien Stormacq · Accepted Answer · 2019-01-31 12:28:28Z

1

The Postgresql COPY command does not support copying data from S3. You'll need to use an intermediate service to download the data from S3 and then upload them to the database.

This doc will help you https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Procedural.Importing.html

This tutorial shows how to use Data Pipeline to move data between S3 and RDS MySQL. Should be fairly similar for Postgress. https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-template-copys3tords.html

answered Jan 31, 2019 at 12:28

Sébastien Stormacq

15k5 gold badges44 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Copy data from S3 file into aws postgresql: invalid argument?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related