Using git submodule to import a python project

Question

(I realise there are a lot of questions on StackOverflow relating to python relative imports; I suspect also that I'm doing this all wrong; but here goes anyway ..)

I have a python project (mysubmod) organised as follows:

/__init__.py
/lib
- /__init__py
- /foobar.py
/models
- /__init__.py
- /hello.py

Importantly:

/models/hello.py imports lib.foobar
/models/hello.py has a main block which runs some tests

I then have a second project (myproj) into which I've imported mysubmod as a git submodule; myproj is now organised as follows -

/mysubmod
/scripts
- /__init__.py
- /__test.py

Where:

scripts/test.py imports mysubmod.models.hello

However when I run /scripts/test.py from the command line, the interpreter now complains that mysubmod/models/hello.py can no longer find lib.foobar.

Reload.

Changed mysubmod/models/hello.py so it now imports ..lib.foobar

Now I can run /scripts/test.py without complaint, but I can no longer run any of the submod scripts as files from the command line; I can only run them as modulles using the python -m option.

Also I can no longer run the submod project files on a standalone basis, even with the -m option, as the interpreter complains about 'Attempted relative import beyond toplevel package'; I have to go up a directory level, ie can only really run mysubmod from the myproj root (where it has mysubmod as the root level directory).

This all seems pretty unsatisfactory. I want to be able to:

use git submodule to import mysubmod into myproj so that it 'just works'
continue to be able to develop mysubmod on a standalone basis (ie run scripts as files not modules)

Does anyone have any views on how to do this ? Any views on best practices with respect to git submodule and python ?

Thanks.

Justin · Accepted Answer · 2013-03-28 07:31:52Z

7

OK the answer is (after many wrong turns)

add a single top level package mysubmod (eg 'helloworld')
with mysubmod imported into myprod as a submodule, symbolically link the helloworld package into the myprod root ('ln -s mysubmod/helloworld helloworld')

This way, the package structure is maintained whether you are working with the submod as a submodule or on a standalone basis.

answered Mar 28, 2013 at 7:31

Justin

4,9136 gold badges41 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

CRISPR Over a year ago

after many years... did you find any better solution?

Justin Over a year ago

use a pip project instead of a git submodule

Georg Plaz · Accepted Answer · 2020-05-17 06:53:07Z

6

You could also add mysubmod to sys.path when you starting your myproj. Then mysubmod/models/hello.py will find lib.foobar.

edited May 17, 2020 at 6:53

Georg Plaz

6,0185 gold badges43 silver badges66 bronze badges

answered Jul 9, 2014 at 12:54

Josh C Josh C

3423 silver badges4 bronze badges

2 Comments

SáT Over a year ago

Yes, this seems like a somewhat nicer solution. This answer provides an example.

Xiaowen Zhang Over a year ago

Where to add the sys.path.append("myproject/mysubmod")? I tried to add it into myproject/scripts/some_script.py, but the submodule still reported the broken imports problem when I executed some_script.py

Collectives™ on Stack Overflow

Using git submodule to import a python project

2 Answers 2

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related