How can I install packages hosted in a private PyPI using setup.py?

Question

I'm trying to write the setup.py install file for a private project, which has both public and private dependencies. The public ones are hosted on PyPI, whereas the private ones are hosted on a server running simplepypi.

I would like both public and private dependencies to be resolved and fetched during installation.

I thus added my dependencies to setup.py:

setup(
    ...
    install_requires = [
        # public dependencies
        'argparse==1.2.1',
        'beautifulsoup4==4.1.3',
        'lxml==3.1.0',
        'mongoengine==0.8.2',
        'pymongo==2.5.2',
        'requests==1.1.0',
        'Cython==0.18',
        # private dependencies
        'myprivatepackage1',
        'myprivatepackage2'
    ],
    dependency_links=['http://pypi.myserver.com/packages'],
    ...
)

I build the package tarball using the command python setup.py sdist and install it in an activated virtualenv using pip install --verbose path/to/tarball.tar.gz.

However, the pip log lines do not mention my private PyPI server anywhere, and https://pypi.python.org/simple/ seems to have been queried twice.

Running setup.py egg_info for package from file:///home/b/code/mapado/mypackage/dist/mypackage-0.5.1.tar.gz
    running egg_info
    creating pip-egg-info/mypackage.egg-info
    writing requirements to pip-egg-info/mypackage.egg-info/requires.txt
    writing pip-egg-info/mypackage.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/mypackage.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/mypackage.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found
    
    reading manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/mypackage.egg-info/SOURCES.txt'
Downloading/unpacking myprivatepackage (from mypackage==0.5.1)
  Could not fetch URL https://pypi.python.org/simple/myprivatepackage/: HTTP Error 404: Not Found (myprivatepackage does not have any releases)
  Will skip URL https://pypi.python.org/simple/myprivatepackage/ when looking for download links for myprivatepackage (from mypackage==0.5.1)
  Could not fetch URL https://pypi.python.org/simple/myprivatepackage/: HTTP Error 404: Not Found (myprivatepackage does not have any releases)
  Will skip URL https://pypi.python.org/simple/myprivatepackage/ when looking for download links for myprivatepackage (from mypackage==0.5.1)
  Could not find any downloads that satisfy the requirement myprivatepackage (from mypackage==0.5.1)
Cleaning up...

What am I missing?

Georg Plaz · Accepted Answer · 2022-09-15 16:35:58Z

21

it looks like you didnt specify your host like the doc of simplepy said you need to setup your ~/.pypirc with the good hostname like

To use it run "simplepypi". You can upload packages by:

[...]

Not using twine yet? Here is the legacy way of uploading Python packages (not recommended):

Modify your ~/.pypirc so it looks like:
[distutils]
index-servers =
    pypi
    local

[local]
username: <whatever>
password: <doesn't matter, see above>
repository: http://127.0.0.1:8000

[pypi]
...

then you'll upload your package on it

python setup.py sdist upload -r local

and could install it from there

pip install -i http://127.0.0.1:8000/pypi <your favorite package>

Hope this will help.

edited Sep 15, 2022 at 16:35

Georg Plaz

6,0185 gold badges43 silver badges66 bronze badges

answered Sep 16, 2013 at 13:33

user1593705

Sign up to request clarification or add additional context in comments.

3 Comments

Balthazar Rouberol Over a year ago

I fixed it! My mistake was that, as you pointed out, I needed to add the "http://127.0.0.1:8000/pypi/" URL to dependency_links. Warning: it needs to be "http://127.0.0.1:8000/pypi/" and not "http://127.0.0.1:8000/pypi", as the latter returns a 404.

Thomas Ruble Over a year ago

Can I do this without modifying ~/.pypirc? That is, just purely through setup.py?

bfontaine Over a year ago

Downvoted because the question is about setup.py. Modifying ~/.pypirc is not an option for me.

r89m · Accepted Answer · 2018-04-13 08:39:05Z

0

dependency_links is ignored by default (at least in pip 9.0.1)

In order for it to reach out to your sever you need to add --process-dependency-links

I believe pip 10 will bring a new mechanism, but for now this has got it working for me

I also had to update dependency_links to include the package name, for example:

dependency_links=[
    "http://internal-pyp:5678/simple/your_package_name"
]

edited Apr 13, 2018 at 8:39

answered Apr 13, 2018 at 6:55

r89m

4254 silver badges6 bronze badges

Comments

yartem · Accepted Answer · 2020-10-22 12:33:14Z

-1

You could make your package as a normal pip package and publish it to the private repo. To install it, you can specify global option --extra-index-url in the config file:

$ cat ~/.pip/pip.conf
[global]
extra-index-url = https://...

answered Oct 22, 2020 at 12:33

yartem

493 bronze badges

1 Comment

DDMC Over a year ago

Be aware of the dependency confusion attack opened by the use of extra-index-url here.

Collectives™ on Stack Overflow

How can I install packages hosted in a private PyPI using setup.py?

3 Answers 3

3 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related