7

The problem

My docker-compose stack is comprised of 'postgresql', 'redis' and 'Python api server' along with a few others like opentracing etc., but the problem area is restricted to the before mentioned.

  1. The entrypoint in my compose file is a shell script that creates a few files and folders dynamically by reading the environment variables amongst the other things it is supposed to do. Now, the creation of these files work like a charm but the file and folder permissions of these dynamically generated files get interesting. On macos, these dynamically generated files and folders are owned by the non-root user who ran docker-compose up. But, on a linux machine running Ubuntu 19.01 these files and folders get owned by root despite Dockerfile explicitly doing a chown non-root-user:non-root-group to the whole project's folder and setting active USER as this non-root-user

  2. The postgres container mounts itself onto given path but the owner of that directory is no longer who created it but some strange systemd-coredump I guess this is because userID and group on Postgres' Dockerfile maps to this username on my linux server? If yes, what is the recommended way to avoid this?

Since the non-root user who ran docker-compose up is unable to retain file and folder permissions on host machine, Im running into permission denied issues. Whilst a chmod 777 helps get away with the problem, I believe chmod 777 never really solves any problem.

Re-iterating that all of this is a problem only on a Linux machine. On Macos running Docker-For-Mac, both pre-existing and dynamically generated files/folders retain the non-root logged in user as their owner and inside the container, the designated USER in Dockerfile remains the owner of all pre-existing (those that get transferred via COPY ) and newly generated dynamic files/folders.

Example

An example of change in file and folder ownership:

drwxrwxr-x 13 sparkle_deployment_2 sparkle_deployment_2 4096 Nov 21 01:00 PROTON/
drwx------ 19 systemd-coredump     docker               4096 Nov 21 01:00 proton_db/

From above, proton_db is where Postgres is supposed to mount onto. This folder was created initially by user - sparkle_deployment_2. After docker-compose up, the owner and group is changed to system-coredump and docker respectively.

Here is a slice of my: docker-compose.yaml

version: "3.4"
services:
  pg:
    container_name: proton_postgres
    restart: always
    image: postgres
    environment:
      - POSTGRES_USER=${PG_USERNAME}
      - POSTGRES_PASSWORD=${PG_PASSWORD}
      - POSTGRES_DB=${PG_TARGET_DB}
    volumes:
      - ${PROTON_POSTGRES_VOLUME_MOUNT}:/var/lib/postgresql/data
    ports:
      - ${PG_TARGET_PORT}:${PG_TARGET_PORT}
  redis:
    container_name: proton_redis
    restart: always
    image: redis
    volumes:
      - ${PROTON_REDIS_VOLUME_MOUNT}:/data
    ports:
      - ${REDIS_TARGET_PORT}:${REDIS_TARGET_PORT}
  proton:
    container_name: proton
    restart: always
    image: proton_stretch
    ports:
      - ${PROTON_TARGET_PORT}:${PROTON_TARGET_PORT}
    expose:
      - ${PROTON_TARGET_PORT}
    volumes:
      - .:/PROTON
      - ${PROTON_SQLITE_VOLUME_MOUNT}:/PROTON/proton-db
    depends_on:
      - pg
      - redis
    entrypoint: ["./proton.sh"]

And, here is my API server's Dockerfile:

FROM python:3.7.3-stretch

RUN apt-get update
RUN apt-get install bash
RUN apt-get install -y gcc g++ unixodbc-dev

RUN groupadd -g proton_user_group
RUN useradd -G proton_user_group default_proton_user

RUN mkdir -p /PROTON
WORKDIR /PROTON
COPY . /PROTON

RUN python3 -m pip install -r requirements.txt --no-cache-dir

RUN chown -R default_proton_user:proton_user_group /PROTON
USER default_proton_user

EXPOSE 3000/tcp

As you see, I'm doing a chown to explicitly have the directory owned by a non-root user. Despite this, when there are files/folders that get dynamically generated, they get root as their owner. And, this happens only on linux.

Win

Like in MacOS, I want the non-root host on linux machine to retain ownership of all pre-existing and dynamically generated files/folders hence not leading to any "permission denied" issues.

Plus the same non-root host on linux machine to retain ownership of the location where Postgres container volume gets mounted onto.

5
  • Did you addgroup and adduser explicitly in your Dockerfile, and set explicit gid and uid respectively? Docker does not do anything special with gid and uid, so if the values in your image are shadowing values of groups and/or users on your host, then ownership will appear as the shadowed groups and/or users. Commented Nov 21, 2019 at 6:57
  • To make this a simpler question to answer, I recommend you ignore/remove the macos part. Docker Desktop augments permissions across OS so your mac user owns the files created in a bind mount. Linux is not that, and just uses native Linux user permissions. Make this question about just fixing your Linux file perms Commented Nov 21, 2019 at 7:33
  • It'll also help to give a reproducible example with dockerfile and compose file. Generally, it's an anti-pattern to bind-mount databases in docker-compose. Try named volumes instead. Commented Nov 21, 2019 at 7:34
  • thanks @BretFisher Totally agree on named volumes. And that is what I am using in my compose yaml. I am more interested in your previous comment of docker-for-mac augmenting permissions. How can we enforce a non-root user and group to docker-compose' containers? I'll update me question with sample of docker-compose and Dockerfile to better reproduce my issue. Thanks for your help. Commented Nov 21, 2019 at 7:47
  • @wmorrell I've updated my question with a portion of my docker-compose.yaml + my API server's Dockerfile. Yes. Im explicitly doing addgroup and adduser as you see above. Thanks for your time and help. Commented Nov 21, 2019 at 7:57

1 Answer 1

8

Since you're bind-mounting the python container in docker-compose, the Dockerfile files and existing permissions are irrelevant. At runtime, it mounts pwd to /PROTON, so anything in the image at /PROTON is hidden and the container only sees the pwd on host.

The user in the container is a simple UID and GID number match to the host. For instance, use id command on host to get your UID and GID. For me, they are 1000 and 1000. You just need to ensure the user and group running in the container are that same UID/GID.

RUN groupadd --gid 1000 proton \
  && useradd --uid 1000 --gid proton --create-home proton

Now that your host user and container user UID/GID are the same, you'll notice that files created in pwd match the usernames of each user. Linux on host will look up the UID 1000 and see its your host user (for me it's bret) and if you do a docker-compose exec proton ls -al /PROTON you should notice it'll lookup user 1000 in the container and see proton. The usernames are just friendly names for the ID's, so just ensure they match between host user and container use and you'll be good.

Unrelated tips:

Sign up to request clarification or add additional context in comments.

2 Comments

Excellent! I see how this is now augmented by docker-for-mac. Thanks Bret! There might be a caveat for this - If a new host logs in, they may not be able to use image built for user with other UID/GID? Is that expected or is there a way to get around that? Thanks again for your time. Really appreciate it
Sure, you'll want both users to belong in a group that has proper access to the files, but none of this is container specific. Containers don't touch permissions. They, like any Linux process, just need to ensure they have either user or group permissions to the files and dirs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.