Postgres Docker: "postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory"

Question

I am having weird issues with official postgres docker image. Most of the time it works fine, if I shut down the container and launch it again, I sometimes get this error but it's not every time:

PostgreSQL Database directory appears to contain a database; Skipping initialization

postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory

I am launching postgres image using this command:

 export $(grep -v '^#' .env | xargs) && docker run --rm --name postgres \
  -e POSTGRES_USER=$POSTGRES_USER \
  -e POSTGRES_DB=$POSTGRES_DB \
  -e POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
  -p $POSTGRES_PORT:$POSTGRES_PORT \
  -v $POSTGRES_DEVELOPMENT_DATA:/var/lib/postgresql/data \
  postgres

I keep variables in .env file, they look like this:

POSTGRES_USER=custom-db
POSTGRES_DB=custom-db
POSTGRES_PASSWORD=12345678
POSTGRES_PORT=5432
POSTGRES_DEVELOPMENT_DATA=/tmp/custom-db-pgdata

When I try to echo variables the values are there so I don't think I'm passing null values to docker env variables.

The directory on my host machine looks something like this:

/tmp/custom-db-pgdata
├── base
│   ├── 1
│   ├── 13407
│   ├── 13408
│   └── 16384
├── global
├── pg_logical
├── pg_multixact
│   ├── members
│   └── offsets
├── pg_notify
├── pg_stat
├── pg_stat_tmp
├── pg_subtrans
├── pg_wal
│   └── archive_status
└── pg_xact

Does the container still start and work or it shuts down with error? — Tarun Lalwani
– Tarun Lalwani, Commented May 6, 2020 at 8:36
The container starts OK but sometimes throws the error at which point its unable to start. — user3056783
– user3056783, Commented May 7, 2020 at 5:33
What do you have in /var/lib/postgresql/data without mounting volume? It seems, that your mounted volume does not contain that conf file — Niklas
– Niklas, Commented May 11, 2020 at 19:10
I am not familiar with docker, there is such kind of things happening with "snap" and apparmor in ubuntu. Due to sandboxing, the application does not see the filesystem available the same way as you and for example what is seen as /tmp directory is a /tmp/somekindofid/... — Tuckbros
– Tuckbros, Commented May 12, 2020 at 6:50
On many Linux distributions the /tmp is not (guaranteed to be) persisted. It may be using tmpfs (in memory), you can see if it is by running df; or it may be cleaned by some run script. — Dario Seidl
– Dario Seidl, Commented May 12, 2020 at 17:21

Maelstrom · Accepted Answer · 2020-05-13 03:47:26Z

3

If it's inconsistent in how it works between executions on the same machine and same session (aka without rebooting) then something isn't mapping your directories properly. Finding what it is that's breaking will be difficult, more so since you're on a Mac. Docker on a Mac you has the extra bonus of running through a VM, so docker is mapping your local drive/path through to the VM and then mapping that into the container image, so there are two different layers where things can go wrong.

Dario has the right idea in his clarifying comments, you shouldn't rely on /tmp since that also has Mac Magic to it. It's actually /var/private/somegarbagestring and is different every bootup. Try switching to a /Users/$USER/dbpath folder and move your data to that, so at least you're debugging with one less layer of magic between data and database.

answered May 13, 2020 at 3:47

Maelstrom

3,0881 gold badge20 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Evandro Coan · Accepted Answer · 2024-05-27 17:04:46Z

I had this problem when I overwrite the default entrypoint of the docker postgres docker image. You should check the Dockerfile source code of your postgres image to see which commands the default entrypoint runs. In my case, I had to write this:

services:
  postgres:
    image: postgres:16.3
    restart: on-failure
    stop_grace_period: 1s
    user: postgres
    environment:
      - POSTGRES_PORT=5432
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    command:
      - /bin/bash
      - -c
      - >
        set -x;
        mkdir -p /tmp/postgresql;
        chown postgres:postgres /tmp/postgresql;
        tail -F /tmp/postgresql/postgresql.log &
        bash /usr/local/bin/docker-enforce-initdb.sh;
        bash docker-entrypoint.sh;
        postgres -c logging_collector=on \
          -c log_directory=/tmp/postgresql \
          -c log_filename=postgresql.log \
          -c log_file_mode=0777 \
          -c log_min_messages=debug5 \
          -c log_min_error_statement=debug5 \
          -c log_statement=all \
          -c debug_print_parse=off \
          -c debug_print_rewritten=off \
          -c log_autovacuum_min_duration=5 \
          -c log_recovery_conflict_waits=on \
          -c debug_print_plan=off \
          -c debug_pretty_print=on \
          -c log_checkpoints=on \
          -c log_connections=on \
          -c log_disconnections=on \
          -c log_duration=on \
          -c log_error_verbosity=verbose \
          -c log_hostname=on \
          -c log_lock_waits=on \
          -c log_parameter_max_length=-1 \
          -c log_parameter_max_length_on_error=-1 \
          -c log_replication_commands=on \
          -c log_temp_files=0
    volumes:
      - "./containerdata/tmp:/tmp"
      - "/etc/localtime:/etc/localtime:ro"

References:

Collectives™ on Stack Overflow

Postgres Docker: "postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory"

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related