← All Articles A Product of Kinsa Creative

Test a Reusable Django Application for Support of Multiple Django Releases with Tox and TravisCI

Instructions for adding Tox and TravisCI to an existing reusable Django application in order to test for backwards and forwards compatibility with multiple Django and Python releases.

There are three main steps to this:

  1. Adding support for testing a reusable Django application against a single version of Python and a single version of Django by using a demo Django project and Setup Test.
  2. Automating the local testing of a reusable Django application against multiple versions of Python and multiple versions of Django in virtual environments using Tox.
  3. Automating the remote testing of a reusable Django application against multiple containerized versions of Python and multiple versions of Django in virtual environments using Tox on the TravisCI platform.

Assumptions

  1. a reusable Django application is already created and hosted on GitHub
  2. a TravisCI account has been created and associated with the GitHub user (TravisCI is free for OpenSource projects)

Create a local copy of the reusable Django application if it doesn't already exist; clone the repo and create a virtual environment to work in:

$ git clone __repo_url__
$ cd $_
$ mkvirtualenv -a . __env_name__

Calling the Test Suite from the Setup script

Since this is a reusable Django application, it needs to be tested in a Django project. Update the project setup script to call the Django test suite. Eric Holser explains this in depth in a still mostly relevant post from 2009. The Django documentation also includes an example.

As Eric explains, running the test suite without the Django management command relies on setting a test_suite in the setup() method call of the project's setup.py file that calls the test runner. That should look something like:

...
setup(
  ...
  test_suite="runtests.runtests",
  ...
)

test_suite is calling a function called runtests in the runtests module which in turn is going to call the test runner. To make that module, create a file called runtests.py also in the root of the project:

$ touch runtests.py

Edit that to look like this:

import os
import sys
import django

from django.conf import settings
from django.test.utils import get_runner


def runtests():
    os.environ['DJANGO_SETTINGS_MODULE'] = 'tests.test_settings'
    django.setup()
    TestRunner = get_runner(settings)
    test_runner = TestRunner()
    failures = test_runner.run_tests(["tests"])
    sys.exit(bool(failures))

if __name__ == '__main__':
    runtests()

The runtests() function calls the test runner for a demo Django project called tests. Now, create that project. This could be done by installing the Django package and creating the project using django-admin createproject tests but that's probably overkill. Instead, create the project from scratch. At the minimum that requires a tests directory in the root of the reusable application, an empty __init__.py to identify it a module, an empty models.py so tests can run, a settings file named test_settings.py, a urls.py file and a test suite (an empty tests.py file for now):

$ mkdir tests; touch tests/__init__.py tests/models.py tests/test_settings.py tests/urls.py tests/tests.py

The Django documentation specifies the minimum requirements for the settings file to be a SECRET_KEY and INSTALLED_APPS list. Depending on the tests, there can be quite a bit more required. The following seems to work after some trial and error:

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

SECRET_KEY = 'fake-key'

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    '__my_reusable_apps_dependency_app__',
    '__my_reusable_app__',
]

ROOT_URLCONF = 'tests.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'APP_DIRS': True,
    },
]

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

In either case, add the name of the reusable app and any dependency apps to the INSTALLED_APPS tuple.

Populate the test application's tests.py file and the urls.py file if required.

Populating tests can be pretty simple, if the existing reusable Django app itself has tests, move those tests into the tests project directory and removed the original file(s). The tests for django-robots, a very simple reusable app on which I based this post, can be viewed on GitHub.

Now the test suite can be called by running $ python setup.py test. This will run the test suite against whatever the active Python version is in the current working environment (Django will be installed according to the requirements in the setup() function, in this example install_requires=['Django>=1.7'], so the latest version of Django will be installed by default.

Tox

When Tox runs, it creates unique virtual environments for each list of dependencies, in this example, different versions of Django. Instead of just testing on whatever active version of Python the development test environment has and whatever version of Django is specified in the setup script, a variety of combinations can be tested against with one command.

Begin by installing Tox:

$ pip install tox

Tox needs a config file called tox.ini. Create this in the root of the project and then open it for editing:

$ touch tox.ini

Taking a quick look at the Django website to find the Supported Versions table to determine which versions of Django need to be supported.

Use the Django FAQ to check the Python requirements for the various Django versions. Django 1.8 requires Python 2.7, 3.2, 3.3, 3.4, or 3.5, Django 1.9 requires Python 2.7, 3.4 or 3.5, and so on.

With that knowledge, the Tox config file should look like:

[tox]
envlist =
    py27-{18,19,110,111}
    py33-{18}
    py34-{18,19,110,111,20}
    py35-{18,19,110,111,20}
    py36-{111,20}
[testenv]
deps =
    18: Django >= 1.8, < 1.9
    19: Django >= 1.9, < 1.10
    110: Django >= 1.10, < 1.11
    111: Django >= 1.11, < 2.0
    20: Django >= 2.0, < 2.1
commands = python setup.py test
skip_missing_interpreters = true

There are two main components, the envlist and the testenv. The envlist includes the versions of Python to be tested against (2.7, 3.2, etc. as defined in the Tox documentation list of environments) and a reference to a variable in the testenv deps below. The testenv deps assigns a variable to a specific Django point release, e.g. 17 references Django >= 1.7, < 1.8. Finally the config calls the test script to run and skip any environments that aren't installed. The skip_missing_interpreters option is useful in development where, for example, Python 3.5 and 3.6 might be installed locally but Python 3.3. and 3.4 are not. Tox will skip over 3.3 and 3.4 rather than failing. When run in the TravisCI environment it might be less ideal since the whole concept of this approach is to test against all the possible environments. Thus, set to true or false as you see fit.

With this in place, tox can be run locally. Note that the system needs to have all the versions of Python listed in the Tox config to test against installed or it will err, although execution will continue if it can find any of the other Python versions. To run Tox locally:

$ tox

TravisCI

TravisCI automates the process of running Tox every time the code gets pushed to Git. Additionally, it handles the installation of all the various versions of Python in their own isolated environments. Because Tox specifies the dependencies, the Travis configuration uses a matrix to specify the version of Python to install and the Tox env config for each environment.

Create the TravisCI configuration file in the root of the project:

$ touch .travis.yml

Edit that:

language: python
matrix:
  include:
    - python: 2.7
      env: TOXENV=py27-18
    - python: 2.7
      env: TOXENV=py27-19
    - python: 2.7
      env: TOXENV=py27-110
    - python: 2.7
      env: TOXENV=py27-111
    - python: 3.3
      env: TOXENV=py33-18
    - python: 3.4
      env: TOXENV=py34-18
    - python: 3.4
      env: TOXENV=py34-19
    - python: 3.4
      env: TOXENV=py34-110
    - python: 3.4
      env: TOXENV=py34-111
    - python: 3.4
      env: TOXENV=py34-20
    - python: 3.5
      env: TOXENV=py35-18
    - python: 3.5
      env: TOXENV=py35-19
    - python: 3.5
      env: TOXENV=py35-110
    - python: 3.5
      env: TOXENV=py35-111
    - python: 3.5
      env: TOXENV=py35-20
    - python: 3.6
      env: TOXENV=py36-111
    - python: 3.6
      env: TOXENV=py36-20
# command to install dependencies
install:
  - pip install tox
# command to run tests
script:
  - tox -e $TOXENV
# containers
sudo: false

The TravisCI docs give an overview of the config. Breaking it down:

  1. the language is defined as python
  2. a matrix of Python versions and corresponding environments is set with each environment defined in the envlist of the Tox file
  3. pip installs tox on the TravisCI container's operating system
  4. tox is called, passing the environment defined above
  5. finally, the file specifies the use of containers rather than sudo

With this in place, TravisCI will run the Tox tests each time a push is made to GitHub:

$ echo ".tox/" >> .gitignore
$ echo ".eggs/" >> .gitignore
$ echo "*.egg-info" >> .gitignore
$ echo "build/" >> .gitignore
$ git add -A; git commit -m "refactored for CI"; git push origin __branch_name__

Revisions

March 20, 2016
Hyperlinked first use of Tox and TravisCI.
August 29, 2016
Added Django 1.10. Removed Django 1.7. Updated links to find supported versions and Python requirements therein. Changed commands to create files to use touch rather than vim. Added commands to populate .gitignore with egg, build, and Tox directories and files.
June 23, 2017
Small edits to improve readability.
December 3, 2017
Updated the Travis config to use a matrix to combine specific versions of Python with the Tox environment. Updated the Tox config to match the Python version to Django version combos set in the Travis config and to add the new configuration parameter, skip_missing_interpreters=true. Added Django 2.0. Ordered the list of revisions chronologically.

Feedback?

Email us at enquiries@kinsa.cc.