30 April 2015

Installing Python 2 and Python 3 Alongside Each Other on Apple Mac OS X with Virtualenv and Virtualenvwrapper

There is a new version of this article which has been revised for development in 2020 using PyEnv, venv, and virtualenv.

Mac OS X comes with a Python installation. Unfortunately it's either somewhat or very out of date depending on how far into Python 3 development you are.

Note: These directions use Vim as the command line editor. Feel free to substitute vim with the editor of your choice. For example, Sublime can also be launched from the command line.

These directions also assume use of the default Bash shell. If you're using ZSH or some other shell, the paths and aliases will be set in your ~/.zshrc profile or whatever file is used to store your other shell's profile rather than ~/.bash_profile.

As a minimum for any Python development I want to have PIP to fetch and install packages from PyPi as well as a virtualenv so that those packages don't get confused with any other versions of them I might be using.

The Python packages from the Python Software Foundation simplify installation by offering a shell updater, PIP installation, GUI applications and Unix command line tools in a one-click install.

Python 2

Installation

Install Python 2 from the latest Python.org package. This allows you to run python2 and pip. After this installation Python 2.x will be accessible at /Library/Frameworks/Python.framework/Versions/2.7/bin/.

Once the Python 2 package is installed, install virtualenv for Python 2 for the User only. When specifying the User installation, Python packages are then accessible at ~/Library/Python/2.7/bin. Specifying the User installation doesn't automatically add virtualenv to the system path which we will do manually in the next step via an alias. This is what allows Python3 and Python2 to run alongside each other:

$ pip install --user virtualenv

To use the virtualenv command, it needs to be on the path, open the shell profile:

$ vim ~/.bash_profile

And append the following, save and quit:

alias virtualenv2='~/Library/Python/2.7/bin/virtualenv'

Source the profile to update the current Terminal window:

$ source ~/.bash_profile

Usage

To create a Python virtualenv, use the virtualenv2 command with a name argument. The name argument is used to make a directory within the current working directory to contain the virtual environment. If you are already within the directory you used for the project, use . instead of a name to create a virtualenv in the current directory:

$ virtualenv2 env_name

And to activate:

$ source env_name/bin/activate

Or, to activate from within the current directory:

$ source ./bin/activate

Then change directory into the virtualenv if you're not already there.

$ cd env_name

Python 3

Installation

Install Python 3 from the latest Python.org package. This allows you to run python3 and pip3. After installation, Python3 will be accessible from /Library/Frameworks/Python.framework/Versions/3.6/bin/.

Once Python 3 is installed, install virtualenv for Python 3 for the user only. Python3 packages are then accessible at ~/Library/Python/3.6/bin:

$ pip3 install --user virtualenv

To use the virtualenv3 command, it needs to be on the path, open the shell profile:

$ vim ~/.bash_profile

And append the following to the bottom, save and quit:

alias virtualenv3='~/Library/Python/3.6/bin/virtualenv'

Source the profile to update the current Terminal window:

$ source ~/.bash_profile

Usage

To create a Python 3 virtualenv:

$ virtualenv3 env_name

And to activate:

$ source env_name/bin/activate

Then change directory into the virtualenv directory:

$ cd env_name

Verification

The above installation makes available the following commands: python to launch Python 2, python3 to launch Python 3, pip to install Python 2 packages, pip3 to install Python 3 packages, virtualenv2 to create a Python 2 virtual environment, virtualenv3 to create a Python 3 virtual environment. With the exception of the aliases for virtualenv2 and virtualenv3, these can be verified using which:

$ which python
/Library/Frameworks/Python.framework/Versions/2.7/bin/python
$ which python2
/Library/Frameworks/Python.framework/Versions/2.7/bin/python2
$ which python3
/Library/Frameworks/Python.framework/Versions/3.6/bin/python3
$ which pip
/Library/Frameworks/Python.framework/Versions/2.7/bin/pip
$ which pip3
/Library/Frameworks/Python.framework/Versions/3.6/bin/pip3
$ ls -al /Library/Frameworks/Python.framework/Versions/2.7/bin/ | grep 'python.*'
... python -> python2
...
... python2 -> python2.7
...
... python2.7

A Full Example

The process is the same for both Python 2 and 3, just the version specified will change. In this case I'll create a Python 3 virtualenv named test_project in my Mac OS X User's Home directory and install the Pandas package.

$ cd $HOME
$ virtualenv3 test_project
$ cd $_

This creates a new directory in my Home directory named test_project. We can verify that by running pwd to see the full path to the working directory, in my case my username is Joe so my Home directory path is /Users/Joe/:

$ pwd
/Users/Joe/test_project

It also sets up some other directories which contain the executables for Python, if we list the items in that directory we can see that:

$ ls -ll
drwxr-xr-x  15 Joe  staff  510 May  1 11:33 bin
drwxr-xr-x   3 Joe  staff  102 May  1 11:30 include
drwxr-xr-x   3 Joe  staff  102 May  1 11:30 lib

To activate the shell, source the activate script in the bin directory. This activates the virtualenv for the current shell session giving priority to the version of Python it is using and packages installed. If you close the Terminal window this session will end and you will be working with the system-wide version of Python unless you activate a different virtualenv.

$ source ./bin/activate

We can verify that we're using the virtualenv by asking which version of Python is being utilized, we'll see the path to the virtualenv's bin in the output:

$ which python
/Users/Joe/test_project/bin/python

We can do the same thing for PIP:

$ which pip
/Users/Joe/test_project/bin/pip

With that knowledge, we can safely install a package such as pandas:

$ pip install pandas

Once that's installed we can verify the installation by listing the packages:

$ pip list
numpy (1.9.2)
pandas (0.16.0)
pip (1.5.6)
python-dateutil (2.4.2)
pytz (2015.2)
setuptools (3.6)
six (1.9.0)

To create our own package, make a directory with an __init__.py script to tell Python it is a package and add the first Python file, for demonstration purposes we'll give the package the name statsanalysis and the script the name utils :

$ mkdir statsanalysis
$ cd $_
$ touch __init__.py
$ touch utils.py

The advantage to creating a package here instead of just creating utils.py is that you modularize your code. By going further and adding and configuring a setup.py file you can create a publishable package that others can use. It also allows you to easily initialize Git to version control the package's files without having to ignore the environment's directories.

Since numpy is installed in the environment you have full access to it:

$ python

Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> exit()

With that knowledge you can edit utils.py to do whatever you need to do with numpy.

Virtualenvwrapper

Virtualenvwrapper provides some nice extras to virtualenv. Assuming Python 3 is the default and desired Python version to be used, install virtualenvwrapper for Python 3 for the user only. This keeps cross contamination with Python 2 to a minimum and allows us to point to the desired installation of virtualenv that virtualenvwrapper should reference. The Python package for the user are accessible at ~/Library/Python/3.6/bin/ which is where you will find virtualenvwrapper.sh after installation which is the shell wrapper that allows the use of virtualenvwrapper's commands:

$ pip3 install --user virtualenvwrapper

And then add the following to the shell profile to use it, setting the user's sites folder as the PROJECT_HOME and storing Python packages at ~/.virtualenvs:

First, open the shell profile:

$ vim ~/.bash_profile

And then add to the bottom:

# Point to virtualenv
PATH="$HOME/Library/Python/3.6/bin:${PATH}"
export PATH

# VirtualenvWrapper
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/sites
export VIRTUALENVWRAPPER_PYTHON=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3
source $HOME/Library/Python/3.6/bin/virtualenvwrapper.sh

Save, quit and source the shell profile or restart the Terminal application to pick up the changes:

$ source ~/.bash_profile

Revisions

May 1, 2015: Added an introduction to explain why I'm installing what I'm installing. Noted that the tutorial requires a basic knowledge of the shell and Vim application. Added a full example.
January 6, 2017: Python 2.7 is now 2.7.13 and Python 3 is now Python 3.6.0. Updated accordingly. Added directions for using virtualenvwrapper. Changed the use of the phrase "installed to" to "accessible at" when referring to executables and Python packages. Noted the option to use Sublime rather than Vim with a link to the directions for using the Sublime command line tool. Removed some preamble language for clarity.
May 5, 2020: Referenced revised and updated article: Installing Multiple Versions of Python Alongside Each Other on Apple Mac OS X with PyEnv

Feedback?

Email us at enquiries@kinsa.cc.