Directory Structure for Importing Python Package using init.py

Question

I've got a python project (projectA), which I've included as a git submodule in a separate project (projectB) as a subfolder. The structure is laid out as

projectB/
  projectB_file.py
  projectA/        (repository)
    projectA/      (source code)
      module1/
        file1.py   (contains Class1)
      file2.py     (contains Class2)
    tests/
      test_file1.py

I'm trying to figure out how to layout __init__.py files so in projectB_file.py I can import Class1 and Class2 as

from projectA.module1 import Class1
from projectA import Class2

I think the "right" answer is to put projectB/projectA into your module search path, then the imports as you've written them should just work. Without the path manipulation, you'll need projectA.projectA.module1 and so on. — Blckknght
– Blckknght, Commented Jan 13, 2016 at 23:17
Ah, hey that does help! I didn't consider that. That's much nicer, as I'm trying to figure out how to make projectA into a proper python package eventually, and wouldnt need that top level __init__.py. I just wish PyCharm would know what I'm doing and tab complete from projectA when using sys.path — abnowack
– abnowack, Commented Jan 13, 2016 at 23:58
Actually marking the top level projectA/ as a source directory in PyCharm also takes care of that issue, so I'm set! If you post your comment as an answer I'll mark it correct — abnowack
– abnowack, Commented Jan 14, 2016 at 0:03

Blckknght · Accepted Answer · 2016-01-14 00:26:19Z

1

I think having the top level projectA part of the import will be a mistake. That will require you to write your imports with projectA duplicated (e.g. import projectA.projectA.module1.file1).

Instead, you should add the top projectA folder to the module search path in some way (either as part of an install script for projectB, or with a setting in your IDE). That way, projectA as a top-level name will refer to the inner folder, which you actually intend to be the projectA package.

answered Jan 14, 2016 at 0:26

Blckknght

106k11 gold badges135 silver badges188 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ethan Furman · Accepted Answer · 2016-01-13 23:26:36Z

0

You'll need an __init__.py in each subdirectory you want to treat as a package. Which in your case means one in:

projectA
projectA/projectA
projectA/projectA/module1
projectA/projectA/tests

It would definitely be better you could lose that first projectA folder.

answered Jan 13, 2016 at 23:26

Ethan Furman

70.1k21 gold badges174 silver badges251 bronze badges

3 Comments

abnowack Over a year ago

Not really sure how to do that, projectA is it's own separate repository. I have the subdirectory to place the actual project code apart from docs/ and tests/ etc. And including init.py everywhere doesn't quite do the trick. As it means including from projectA.projectA.module1.file1 import Class1

Ethan Furman Over a year ago

@abnowack: If projectA is a separate project, why do you have it under projectB?

abnowack Over a year ago

projectA is a generic library for interpreting a dataset I've written, projectB is a specific analysis I am doing for a paper.

justengel · Accepted Answer · 2016-01-14 13:36:02Z

This is an annoying issue. The main approach I use is to edit the PYTHONPATH. This is essentially how I do it.

ProjectB/                   # Main Repository
    projectb/               # Package/library
        __init__.py
        projectB_file.py
    docs/
    tests/
    setup.py                # Setup file to distribute the library
    freeze.py               # CX_Freeze file to make executables

    ProjectA/               # Subrepository
        projecta/           # Package/library
            __init__.py
            projectA_file.py
            file2.py
            submodule/
                __init__.py
                file3.py
        pa_module1/         # Additional package/library in 1 project
            __init__.py
            file1.py
        docs/
        tests/
        setup.py

With this setup I have to edit the python path before running ProjectB's code. I use this structure, because it is easier for deployment and distribution. The subrepository helps track the version of ProjectA for each release of ProjectB.

cd ProjectB
export PYTHONPATH=${PWD}:${PWD}/ProjectA
python projectb/projectB_file.py

projectB_file.py

from projecta import projectA_file

This is for my development environment. For a release environment someone would install with the below code.

# Need a correct setup.py to handle 2 packages for pa_module1 and projecta
pip install ProjectA (install projecta to site-packages) 
cd ..
pip install ProjectB (install projectb to site-packages)

This means projectb and projecta are in site packages. From there any file can simply

from projectb import projectB_file
from projecta import projectA_file, file2
from pa_module1 import file1

# Personally I don't like imports that use project.sub.sub.sub ...
# I have seen it cause import confusion and problems, but it is useful in some situations.
from projecta.submodule import file3

#import projecta.projectA_file
#import projecta # if __init__.py has import code

file1.Class1()
file2.Class2()
file3.Class3()

Additionally with pip you can install a library as a developer environment which links to the real project directory instead of copying files to site-packages.

pip install -e ProjectA

Collectives™ on Stack Overflow

Directory Structure for Importing Python Package using init.py

3 Answers 3

Comments

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related