Practical PyPI package creation

This is by no means a 100% complete guide for creating PyPI package. This is basically compilation of this article, official docs and useful tips you might need while creating your package.

Registration

Register at PyPi live and at PyPI test. PyPI live is where your package is indexed at, while PyPI test is there just to check that your package is build correctly.

.pypirc

To make your life easier, you can create ~.pypirc config what will be used for accesing PyPI sites.

[distutils]
index-servers =
 pypi
 pypitest

[pypi]
repository=https://pypi.python.org/pypi
username=your_username
password=your_password

[pypitest]
repository=https://testpypi.python.org/pypi
username=your_username
password=your_password

Package preparation

Create setup.py file. This file is the main source of info about your package, including subpackages, executable files etc.

from distutils.core import setup
from setuptools import find_packages

setup(
  name='project_name',
  packages=find_packages(exclude=['docs', 'doc', 'test']),
  version='0.1.2',
  description='Your Awesome Package',
  author='your_username',
  author_email='email@mail.com',
  url='https://github.com/user/package', # link to the GitHub sources
  download_url='https://github.com/user/package/arhive/tag.tar.gz', # Where to dowload your package
  keywords=['python', 'cool'],
  scripts=['bin/awesome_package', 'bin/awesome_package_util'], # List of executables of your project
  classifiers=[],
  install_requires=['package1', 'numpy >= 1.9.1'] # List of your dependencies
)

Packages

packages=find_packages(exclude=['docs', 'doc', 'test'])

Is this line we use `find_packages` util to automaticaly include all the subpackages in you package instead of listing them by hand. Also, we are excluding docs, doc and test directories as out package works without them.

Download url

download_url='https://github.com/user/package/arhive/tag.tar.gz', # Where to dowload your package

Download url is where PyPi will download your package from. Make sure to tag your releases as you update a version

git tag 0.1.2 -m "0.1.2 version"
git push --tags origin master

After this the new release can be found at https://github.com/{username}/{module_name}/archive/{tag}.tar.gz.

Scripts

scripts=['bin/awesome_package', 'bin/awesome_package_util'], # List of executables of your project

In this line list all your executables. After the installation you can use them as any other «binary» file

awesome_package
awesome_package_util

Note that in this files, you cannot do relative imports like

from ..project_name import *

As your package is already installed, import it as global package

import project_name

Also note that the first line of every executable should be

#!/usr/bin/env python

Install requires

install_requires=['package1', 'numpy >= 1.9.1'] # List of your dependencies

install_requires is a list of dependencies of your package. You can specify name of the dependency and version that should be used.

Setup.cfg

Now you will need setup.cfg that list additional info about your package.

[metadata]
description-file = README

Now, the problem is — PyPI doesn’t understand MarkDown. So we need to transform it to something else, for example reStructuredText. For that, we can use pandoc

sudo apt-get install pandoc

Not we can transform README.md to README (in reStructuredText format)

pandoc --from=markdown --to=rst --output=README README.md

Don’t forget to add README to .gitignore

Testing

Before uploading your package, make sure that everything is okay localy. You can install your package from sources using

pip install . --user --upgrade

—user to install it to ~/.local, —upgrade to update it. Without —upgrade pip will complain about package already been installed.

Now you can cd to some other directory, run python and execute

import project_name

If you have executable files, you can run executable_name to check if they are installed correctly.

If you don’t want to run pip instal . every time you change something in your package, you can run

pip install -e . --user

This will install the project in editable mode, basically creating symlink to your project. After you’re done with the changes, you can come back and run pip install . —user —upgrade to check if everything is working correctly.

PyPI test

As pointed out by a Reddit user, sdist upload is not secure, as it uses plain HTTP, which can lead to your credentials stolen. Thus we will use twine as a tool for uploading our package

First you need to install twine

sudo apt install twine

Lets build our package

python setup.py sdist

This will create dist/{package-name}-{tag}.tar.gz archive that we now can download to PyPI.

But first letf publish your package to PyPI test to check if everything build correctly

This command will register your package

python setup.py register -r pypitest

This command will publish your package. You need to replace {package-name} and {tag} with the name of your package and the tag you are currently publishing.

 twine upload dist/{package-name}-{tag}.tar.gz -r pypitest

PyPI live

If the previous step went without any errors, you can publish your package to the PyPi live

Register your package

python setup.py register -r pypi

Upload it

twine upload dist/{package-name}-{tag}.tar.gz

And that is it! Everyone can pip install your awesome package.

Post install script

If you need to run some commands after the main installation is done, you can write a function that will be executed after setup is finished.

Add to setup.py before setup() call

from setuptools.command.install import install
import atexit
import os
import sys

name = 'cool_math3'

class CustomInstall(install):
  def run(self):
    atexit.register(self.post_install)
    install.run(self)

  def find_installation_path(self):
    for p in sys.path:
      if os.path.isdir(p) and name in os.listdir(p):
        return os.path.join(p, name)

  def post_install(self):
    '''
    This function is called after setup is done
    '''
    installation_path = self.find_installation_path()

    print(installation_path)

Here we create class for custom installation. post_install method will be called after installation is done. installation_path will point to the directory that contains installed project.

And add cmdclass parameter to setup() call

    setup(
        name=name,
        ...
        cmdclass={'install': CustomInstall,},
        ...
    )

To actually see that print(…) in post_install you need to run pip with -v flag

pip3 install . --user --upgrade -v

Please note that post_install is mostly a hack and should be used with great care. If you can avoid it, use more traditional tools.

Final notes

You might notice that we only upload the package to the index once. But it is available for both Python 2 and Python 3 to install.