Releasing a GitHub-Hosted, GitFlow-Release-Managed, Wheel-Packaged Python Package with a Markdown-formatted Readme to PyPI Using Twine

Last updated 25 February 2024.

Registering a PyPi Account

Before you do anything else, you will need accounts on the test and production PyPi websites:

1. Navigate to the Test PyPi site and create an account for test releasing
2. Navigate to the PyPi site and create an account for officially releasing

Once you have accounts set up, you will be able to create a token for authentication which you will store in a .pypirc file.

1. To include both the production and test PyPi sites, the .pypirc file will look like the following (swapping out your own token for my-token and my-test-token where indicated:

[distutils]
index-servers =
    pypi
    pypitest

[pypi]
repository: https://upload.pypi.org/legacy/
username: __token__
password: my-token

[pypitest]
repository: https://test.pypi.org/legacy/
username: __token__
password: my-test-token

Using a Markdown-formatted Readme

Note that a Markdown-formatted Readme file is possible but PyPi won't display the contents as the long description if the Readme file is being used as the long description. This is a nicety of PyPi which provides a browsable interface to extra information about a package.

To use a Markdown-formatted Readme file, assuming it is in the root of the project (analogous to the setup.py script) in the setup.cfg, add:

[metadata]
description-file = README.md

Then, in the setup.py script, the call to setup() will not include a kwarg for long_description. As opposed to an instance using a ReStructuredText-formatted Readme where that kwarg argument would be long_description=open("README.rst").read(),.

Creating a Development Environment for a Python Package and Uploading it to PyPI

Assuming you've created a .pypirc configuration file and registered for PyPI Live and PyPI Test as described above, the directions for using twine rather than sdist are pretty much a drop-in-place replacement.

The directions that follow assume that you have a project you've built and have been hosting on GitHub and are ready to release to PyPi.

Starting from the top, we will create a new virtual environment, clone in the repo, run tests and lint the project, register the package to PyPi Test and PyPi Live and then upload.

Clone the Repo

$ git clone git@github.com:myuser/myproject.git

This creates a directory tree as follows:

├── myproject
    ├── LICENSE.txt
    ├── README.md
    ├── myproject
    │   ├── __init__.py
    ├── setup.cfg
    ├── setup.py
    └── tests
        ├── __init__.py
        └── test_myproject.py

Create a Virtual Environment

Using venv:

$ cd myproject
$ python -m venv .venv
$ source .venv/bin/activate

Install the Package for Development

$ pip install -e .

Install Wheel, Twine and PyLint

$ pip install wheel twine pylint

Run the Tests

$ python ./tests/test_myproject.py
$ pylint ./myproject 

Setup Git Flow

I'm using Nvie's Git Flow to manage releases to GitHub. The first time the project is cloned and setup, GitFlow needs to be initialized.

By default my project clones the develop branch which is my default working branch. Git Flow uses master to track the latest release. This branch is also on GitHub so it needs to be checked out.

$ git checkout master

Once the master branch exists alongside develop, Git Flow can be initialized with the defaults.

$ git flow init -d

Create the Distribution

This needs to be done each time the version (or subpoint version) gets bumped.

Tag the version for release.

$ git flow release start versionNumberAndSubPoint

Edit setup.py and bump the version and download_url versions (my download_url points to a specific tarball of a tag on GitHub).

$ git flow release finish versionNumberAndSubPoint

Push everything back to GitHub.

$ git push --all; git push --tags

Create the dist.

$ python setup.py sdist

Create the Wheel

Depending on whether the package is universal to Python 2 and 3 or not changes how this is run. Instructions can be found in the Python Packaging User Guide. To create a Universal package that works with both Python 2 and 3:

$ python setup.py bdist_wheel --universal

Register the package to PyPI Test

This only needs to be done the first time.

$ python setup.py register -r pypitest

Upload to PyPI Test

$ twine upload dist/* -r pypitest

Register the package to PyPI Live

This only needs to be done the first time.

$ python setup.py register -r pypi

Upload to PyPI Live

$ twine upload dist/*

Additional Resources

There are a number of existing tutorials and articles on this subject. However, many tutorials and directions cover eggs and easy install rather than pip and wheel, or use sdist rather than twine. The Python Packaging and Users Guide is the actual, most-up-to-date documentation on the process of releasing a package to the Python Package Index (PyPI). It covers the additional parameters of the setup file including the more complicated variables of classifiers and possible values therein. It also includes instructions for uploading files using the better practice of using twine instead of sdist as well as the wheel binary package distribution format intended to replace eggs.

The Python Project Howto, while aging, does a good job of covering general best practices before deciding to release a module.

How to submit a package to PyPI provides a good basic overview of what to include in the setup.py script and how to link to documentation when it is in a markdown-formatted readme file. This also provids all the directions for creating PyPI Live and PyPI Test accounts and a .pypirc configuration file. The official documentation is contained in the distutils documentation. The basic steps are:

Revisions

August 20, 2015

Revised virtualenv instructions to utilize virtualenvwrapper. Added instructions for using wheel. Added GitFlow instructions.

February 23, 2016

Noted that Universal wheels support both Python 2 and Python 3; consolidated installation of Python packages necessary for linting and pushing to PyPi.

March 20, 2017

Added directions regarding Markdown formatted readmes as well as the .pipyrc file.

June 11, 2017

Revised the language in the introduction of the article to be more straightforward and moved it to a resource section at the bottom of the article.

February 25, 2024

Updated the link for configuring a .pypirc file and updated the notes surrounding the configuration of the .pypirc file to reference a token rather than a username and password. Switched from using VirtualEnvWrapper for the virtual environment creation to the Python 3 native venv tool.

Feedback?

Email us at enquiries@kinsa.cc.