Customizing python package directory layout with setup.py

Question

Suppose I have the following directory structure:

src/
└── python/
    └── generated/
        ├── __init__.py
        ├── a.py
        └── lib/
            ├── __init__.py
            └── b.py

What does my setup.py need to look like in order to create a dist with a directory layout like:

src/
└── python/
    ├── __init__.py
    ├── a.py
    └── lib/
        ├── __init__.py
        └── b.py

The goal is to simply eliminate the generated folder. I've tried endless variations with package_dir and can't get anything produced other than the original directory structure.

Do you see any error or it basically does not output anything? Can you please highlight what happens with the execution of setup? — Nagaraj Tantri
– Nagaraj Tantri, Commented Sep 8, 2019 at 5:15
Please list all the combinations you have tried for package_dir ? — Tarun Lalwani
– Tarun Lalwani, Commented Sep 8, 2019 at 11:21
@TarunLalwani Many were experiments in the moment that were immediately discarded after failure. The records are lost. User sinoroc provided a great solution if you are interested. — Tyrick
– Tyrick, Commented Sep 10, 2019 at 16:54

sinoroc · Accepted Answer · 2023-08-16 17:06:20Z

6

+50

In this specific case, for this specific project directory structure, the setup.py script should be placed in the src directory and should look like this:

#!/usr/bin/env python
    
import setuptools
    
setuptools.setup(
    name='Thing',
    version='1.2.3',
    packages=[
        'python',
        'python.lib',
    ],
    package_dir={
        'python': 'python/generated',
    },
)

Note the package_dir setting. It instructs setuptools to get the code for the python package from the directory python/generated. In the built distributions you will then find the right directory structure.

edited Aug 16, 2023 at 17:06

answered Sep 9, 2019 at 11:44

sinoroc

23.4k4 gold badges59 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Tyrick Over a year ago

Placing the setup.py inside the src directory was not an option for me. But I was about to get the desired result with the same principle: package_dir: { 'src.python': 'src/python/generated' } and adjusting the packages! Thanks a ton!

sinoroc Over a year ago

Glad it helps. Note that you could do package_dir: { 'whatever_package_name_you_like': 'src/python/generated' }, and adjust the packages argument accordingly.

FaeLudic · Accepted Answer · 2019-09-09 11:29:59Z

First, here is my solution:

#!/usr/bin/env python

import os, shutil
from setuptools import setup
from setuptools.command.build_py import build_py

class BuildPyCommand(build_py):
  """Custom build command."""

  def run(self):
    shutil.rmtree('src.tmp', ignore_errors=True)
    os.mkdir('src.tmp')
    shutil.copytree('src/python/generated', 'src.tmp/python')
    build_py.run(self)

setup(cmdclass={ 'build_py': BuildPyCommand },
      name='Blabla',
      version='1.0',
      description='best desc ever',
      author='Me',
      packages=['python', 'python.lib'],
      package_dir={'': 'src.tmp'},
      setup_requires=['wheel']
     )

And you can generate your distribution with:

python setup.py build bdist_wheel

The idea is perform a two steps build:

I generate a valid source structure
I build this temporary structure

And I deliver it in a wheel because it doesn't require future users to understand my trick. If you give it a try with a source distribution, you will notice that you need to publish the generated files as data (not difficult, but troublesome, and, I guess you will want to hide your tricks from your users).

But, I think that there is a design flaw in your process. The file src/python/generated/__init__.py, assumed to be a module <something>.generated eventually becomes your <something>.python, which is troublesome. It would be much simpler and more robust to generate a valid Python structure: src/generated/python/__init__.py. The setup.py would become trivial and your generator wouldn't be more complex.

Collectives™ on Stack Overflow

Customizing python package directory layout with setup.py

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related