Reusing postgresql database from volume in docker-compose

Question

When I created volume in Docker using command:

docker volume create pg-data

Then I set up basic postgresql database from postgres image:

docker run --rm -v pg-data:/var/lib/postgresql/data --name pg-docker -e POSTGRES_PASSWORD=docker -p 5433:5432 postgres

Everything worked fine. Database persist and I can even access it directly from the host. I created several roles here like app_user_1.

Now then I wanted to spin up postgresql in container using docker-compose. I shutdown the above postgresql container beforehand.

There I have this settting:

version: '3.7'

services:
  db:
    image: postgres
    volumes:
      - pg-data:/var/lib/postgresql/data/
    expose:
      - 5432
    restart: always
    environment:
      - POSTGRES_PASSWORD=docker
      - POSTGRES_USER=postgres
  web:
    build: .
    volumes:
      - ./app:/app
    ports:
      - 8001:8000
    environment:
      - ENVIRONMENT=dev
      - TESTING=0
    depends_on:
      - db

volumes:
  pg-data:

However it seems that even though I mapped the same volume and used same env settings as in docker run command the postgresql instance in container created with docker-compose has no databases and no roles at all. I get the following error:

psql: error: FATAL: role "postgres" does not exist

or

psql: error: FATAL: role "app_user_1" does not exist

So it seems it behaves as though as it is different instance of postgresql. When I restarted the first container with docker run everything was there (all the databases and roles).

Any idea why this is happening? How can I reuse the databases from the first container in the docker-compose?

Dan Streeter · Accepted Answer · 2021-07-14 07:04:27Z

2

You need to define the volume you wish to use (the one you created manually with docker volume create as external to docker-compose as it was created externally

This is because the volumes created by docker-compose are 'internal' to it, so using ones created by just docker are 'external'. =)

Ref the offical docs at https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose

The change to your compose file would be as follows:

...
volumes:
  pg-data:
    external: true

(Just that last line)

Hope that helps! =)

Additional Note You can confirm this, by performing a docker volume ls | grep pg-data command which will list all volumes, then only show you the ones referencing 'pg-data'. On my system where I was testing before I gave my answer, I get the following:

docker volume ls | grep pg-data
local     pg-data
local     postgresstackoverflow_pg-data

As you can see, the docker volume create one is listed first, as a local volume called 'pg-data', then the docker-compose.yml created one is next prefixed by the naming convention of docker-compose with the directory name that it was in at the time.

edited Jul 14, 2021 at 7:04

answered Jul 14, 2021 at 6:59

Dan Streeter

2,4112 gold badges20 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Alex T Over a year ago

Great explanation. Are there any drawbacks of using external volumes in docker-compose?

Dan Streeter Over a year ago

Erm - as an instant thought, not really - no... other than the obvious points like having them managed within the yaml means they are instantly identifiable, and you may not immediately manually purge one called 'db-data' by accident as 'my-really-important-project-db-data" is clear as to what it is... I dont 'think' that there are any restrictions of using a yaml defined volume outside (as an external) one and if you define it within one yaml, you MIGHT be able to use it as an externally defined volume in another - i'd have to run that use case through with a POC... dunno

Alex T Over a year ago

I can see that the volume created by yaml persists outside, so it should not be a problem to mount it in another dockerfile.

Dan Streeter Over a year ago

Super - I think the only concern I can think of with that is to ensure that you / other engineers working within that environment know its a shared volume outside that compose file... This is where I would probably approach it by creating as an external volume to start with docker volume create.... and then define it in both compose yamls as external - so its clear from within the yaml that its outside... and a volume created manually is reasonably obvious. You also wouldnt be able to loose it with docker-compose down -v (which deletes the volumes as well)

Collectives™ on Stack Overflow

Reusing postgresql database from volume in docker-compose

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related