In testing.postgresql, cannot find initdb command inside Docker

Question

There is a similar thread that has the same problem but wasn't able to work for me. Basically, I'm trying to unittest myproject with testing.postgresql and i'm running it inside a docker container.

However, I get a RuntimeError: command not found: initdb when I run my command,

docker-compose exec myproject python3 -m unittest

This works in WSL but not inside docker.

Looks like it's not able to find initdb. Is it having conflict issues with the database created in docker-compose.yml or am I missing some lines inside the docker files?

I outline my test like example from the docs:

class TestPostgresqlInteraction(unittest.TestCase):

    Postgresql = testing.postgresql.PostgresqlFactory(cache_initialized_db=True)

    with testing.postgresql.Postgresql() as postgresql:
        engine = create_engine(postgresql.url())
        dbConn = psycopg2.connect(**postgresql.dsn())
        cursor = dbConn.cursor()
        def setUp(self):
            self.postgresql = self.Postgresql()

        def tearDown(self):
            self.postgresql.stop()

        def testMethod(self)
            conn = psycopg2.connect(**self.postgresql.dsn())
            cursor = conn.cursor()
            # execute cursor and do tests... 

        def tearDownModule(self):
            self.Postgresql.clear_cache()

docker-compose.yml

version: "3"
services:
    myproject:
        build:
            context: .
            dockerfile: ./myproject/Dockerfile
        depends_on:
            - database
        volumes:
            - ./myproject:/code
        stdin_open: true
        tty: true
    database:
        build:
            context: .
            dockerfile: ./database/Dockerfile
        environment:
            POSTGRES_USER_FILE: /secrets/dbUser.txt
            POSTGRES_PASSWORD_FILE: /secrets/dbPassword.txt
        ports:
            - "8765:5432"
        volumes:
            - otherdata:/var/lib/postgresql/data

myproject/Dockerfile

FROM python:3.7.3-stretch

RUN apt-get update && apt-get install -y \
    poppler-utils

ARG moduleDir=myproject

WORKDIR /code

COPY secrets/ /secrets

# COPY $moduleDir/myproject/ ./

COPY $moduleDir/requirements.txt requirements.txt

RUN pip install -r requirements.txt

database/Dockerfile

FROM postgres:11.3

WORKDIR /secrets

COPY secrets/dbUser.txt dbUser.txt
COPY secrets/dbPassword.txt dbPassword.txt

RUN mkdir -p /docker-entrypoint-initdb.d

COPY database/schema.sql /docker-entrypoint-initdb.d

Traceback:

ImportError: Failed to import test module: tests.testMyproject
  File "/usr/local/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/code/tests/testmyProject.py", line 40, in <module>
    class TestPostgresqlInteraction(unittest.TestCase):
  File "/code/tests/testmyProject.py", line 46, in TestPostgresqlInteraction
    Postgresql = testing.postgresql.PostgresqlFactory(cache_initialized_db=True)
  File "/usr/local/lib/python3.7/site-packages/testing/common/database.py", line 52, in __init__
    self.cache = self.target_class(**settings_noautostart)
  File "/usr/local/lib/python3.7/site-packages/testing/common/database.py", line 92, in __init__
    self.initialize()
  File "/usr/local/lib/python3.7/site-packages/testing/postgresql.py", line 50, in initialize
    self.initdb = find_program('initdb', ['bin'])
  File "/usr/local/lib/python3.7/site-packages/testing/postgresql.py", line 144, in find_program
    raise RuntimeError("command not found: %s" % name)
RuntimeError: command not found: initdb

Michał Zaborowski · Accepted Answer · 2019-09-24 15:38:16Z

0

Using testing.postgresql means that you will get new instance of PG prepared by the tool - as it is described in their docs. What you need to do is to point to that other docker. Simply use psycopg and connect to database:

dsn = "dbname='" + RDS_DBNAME + "' user='" + RDS_USERNAME + "' host='database' password='" + RDS_PASSWORD + "'"
dbConn = psycopg2.connect(dsn)

Besides that - you with looks strange. But that can be only my impression.

answered Sep 24, 2019 at 15:38

Michał Zaborowski

4,4032 gold badges23 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

resolute Over a year ago

Is RDS_DBNAME, RDS_USERNAME, RDS_PASSWORD referring to what I defined inside secrets/ directory?

Michał Zaborowski Over a year ago

@resolute - yes RDS_DBNAME is name of you database created by schema.sql, and to get there you need credentials, user from dbUser.txt, and password from dbPassword.txt :)

David Vasconcelos · Accepted Answer · 2020-02-11 01:21:17Z

0

Your Docker container may lack the which shell command.

testing.postgresql calls testing.common.database.get_path_of, which then executes the which command to locate the initdb program:

def get_path_of(name):
    if os.name == 'nt':
        which = 'where'
    else:
        which = 'which'
    try:
        path = subprocess.Popen([which, name],
                                stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE).communicate()[0]
        if path:
            return path.rstrip().decode('utf-8')
        else:
            return None
    except Exception:
        return None

If which is missing, you can probably add it using a package manager. This worked on my CentOS-based container:

yum install which

answered Feb 11, 2020 at 1:21

David Vasconcelos

642 bronze badges

1 Comment

yogender Over a year ago

yum install which does not work for me

kriss · Accepted Answer · 2020-11-17 21:36:55Z

0

To use testing.postgresql you need access to postgresql server in your myproject Docker not just in the database Docker. The local temporary test database will connect to the other database to get data, but it still means you need a local instance of postgresql server inside the myproject.

In your myproject/Dockerfile just change the first RUN and add postgresql after poppler-utils.

answered Nov 17, 2020 at 21:36

kriss

24.3k17 gold badges104 silver badges120 bronze badges

Collectives™ on Stack Overflow

In testing.postgresql, cannot find initdb command inside Docker

3 Answers 3

2 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related