Docker execute can't find file within container

Question

I've set up a container to run some python scripts, but when I attempt to run them from the host with 'docker exec' it throws an error -

OCI runtime exec failed: exec failed: container_linux.go:370: starting container process caused: no such file or directory: unknown

but, if I open a shell to the container, I can see my script is clearly there, and can I can even run it.

DockerFile:

FROM python:3.7.1

WORKDIR /regex_locator
COPY ./py_code/req.txt ./proj/req.txt
RUN pip install -r ./proj/req.txt

CMD ["/bin/bash"]

Makefile:

    MAJOR?=0
MINOR?=1

VERSION=$(MAJOR).$(MINOR)

#vars
IMAGENAME=regex_locator
IMAGEFULLNAME=${IMAGENAME}-${VERSION}
CONTAINERNAME=${IMAGEFULLNAME}-TESTCONT

.PHONY: help build run all

help:

        @echo ""
        @echo "Makefile commands:"
        @echo "build"
        @echo "push"
        @echo "all"

.DEFAULT_GOAL := all

build:
        @docker build -t regex_locator-img .

run:
        @docker run -dit --name regex_locator-test_cont regex_locator-img
        @docker cp ./py_code/  regex_locator-test_cont:/regex_locator/proj/
        @docker exec -it regex_locator-test_cont /regex_locator/proj/py_code/run_tests.py


all: build run

Thanks in advance!

How have you verified that the file is in the image? Can you show an example of running it successfully? — larsks
– larsks, Commented Mar 20, 2021 at 16:15

David Maze · Accepted Answer · 2021-03-20 16:23:12Z

1

I'd recommend using an ordinary Python virtual environment here. This uses tooling built into the standard Python distribution (so it's available on almost every Unix-like host; Docker would be an additional installation) and doesn't require administrator-level permissions. Since this uses the host filesystem normally, you don't need to copy files around, manually map directories, worry about wrong-architecture native libraries, and so on.

VENV := venv

# Create the virtual environment
$(VENV)/bin/python3:
        python3 -m venv "$(VENV)"

# Any application stub installed by a package
$(VENV)/bin/pytest: requirements.txt $(VENV)/bin/python3
        "$(VENV)/bin/pip" install -r "$<"

# Run the tests
.PHONY: test
test: $(VENV)/bin/pytest
        "$(VENV)/bin/python3" ./py_code/run_tests.py

If you need to build a Docker image, it generally should be self-contained. Do not bind-mount or docker cp the code into the image; COPY it in, so that it can be docker run without also having a copy of the source code separately. You can then docker run an alternate command, without needing the docker exec debugging tool.

REGISTRY := docker.io
IMAGE := me/regex-locator
TAG := latest
DOCKER_IMAGE := $(REGISTRY)/$(IMAGE):$(TAG)

.PHONY: docker-build docker-push docker-run docker-test

docker-build: .docker-build
.docker-build: Dockerfile requirements.txt app/main.py ...
        docker build -t "$(DOCKER_IMAGE)" .
        touch "$@"

docker-push: docker-build
        docker push "$(DOCKER_IMAGE)"

docker-run: docker-build
        docker run --rm -p 8000:8000 "$(DOCKER_IMAGE)"

docker-test: docker-build
        docker run --rm "$(DOCKER_IMAGE)" ./proj/run_tests.py

This requires you to make sure your image actually is self-contained -- it shouldn't be an empty shell that could eventually run something.

FROM python:3.7.1

WORKDIR /regex_locator
COPY ./py_code/req.txt .  # no particular need for a subdirectory
RUN pip install -r req.txt
COPY ./py_code .          # make sure to copy the rest of the application too
CMD ./main.py             # and set a useful command for the container to run

Your unit tests shouldn't have complex dependencies. (Use an in-memory database like SQLite or mock other dependencies: tests that are 80% functional and 100% easy to run are much better than tests that are 100% functional but very hard to run.) It shouldn't make a different whether they run in the container or in a virtual environment. In that case you can run unit tests on the host, and if they pass, build and publish your image.

make test         # in a virtual environment
make docker-push  # building the image along the way

answered Mar 20, 2021 at 16:23

David Maze

165k47 gold badges262 silver badges302 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Adi_Zav Over a year ago

Hey David, thanks for the thorough answer. Thing is, I don’t want to copy my scripts when the container’s built, because I don’t want to have to build new containers whenever I update my code. Is there another way to pass files to the container? Other than using git or any alternative

David Maze Over a year ago

If you use a non-Docker virtual environment, you don't need to copy anything anywhere. Otherwise, rebuilding the image when the code changes is routine (and inexpensive for Python apps, it's not actually doing anything more than the docker cp you have in the question).

Adi_Zav Over a year ago

Thing is, the container is going to be used for testing purposes, not a production environment. Is there no solution for updating the code files when running the container, instead of building it over and over again?

Collectives™ on Stack Overflow

Docker execute can't find file within container

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related