Python Development with Virtual Environments

From BC Wiki
Jump to: navigation, search

Topics to cover:

  • Pip
  • Virtualenv
  • Virtualenvwrapper

Python Install Python (PIP)[edit]

You can use pip freeze or pip list to generate a list of the Python packages installed in your current environment. You can output this list to a file and use that file to install the exact same packages in another environment.


See existing environments—and switch between them—with workon (has tab completion).

Make new environments with mkvirtualenv.

Once in a virtual environment, use this to update all Python packages:

pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U

Development Environment Examples[edit]

All of these examples start from a completely fresh copy of the listed operating system. I create and test these configurations, and do most of my development and troubleshooting work, by cloning a virtual machine running fully-up-to-date Ubuntu 16.04 with the user interface configured exactly how I like it; the cloned VM uses a linked disk tied to the parent VM that only records changes, so I can create a new clean VM to experiment with in just a few seconds and it doesn't take up very much hard drive space.

Ubuntu 16.04 - Scientific Computing[edit]

Prepare the virtual environment tools:

sudo apt install python-pip
pip install --upgrade pip
pip install virtualenvwrapper
nano ~/.bashrc
# add this to the end of the file:
  # setup for virtualenvwrapper
  export WORKON_HOME=$HOME/.virtualenvs
  export PATH=$PATH:$HOME/.local/bin
  source /home/brandon/.local/bin/
source ~/.bashrc 

set up a basic virtual environment:

mkvirtualenv py2sci
pip install requests[security]
pip install numpy scipy jupyter

Now we have to install some system-wide stuff so that we can use the various plotting backends in matplotlib. We have to use `apt install` instead of `pip install` because these things are not Python packages.

Installing wxpython_phoenix should get wx-based backends working:

pip install --upgrade --trusted-host --pre -f wxPython_Phoenix

Installing pyside should get qt4-based backends working:

sudo apt-get install build-essential cmake libqt4-dev libxml2-dev libxslt1-dev python-dev qtmobility-dev python-pip

pip install pyside

Installing cairocffi should get cairo-based backends working:

pip install cairocffi

Installing tkinter and the development headers should get tk-based backends working:

sudo apt-get install python-tk tk-dev

Now you can install matplotlib, either using pip:

pip install matplotlib

or from source pulled from Github:

sudo apt-get install dvipng
git clone
cd matplotlib
python install

If you'd like the gtk3-based backends to work as well, you'll need to install some stuff system-wide; after you've copied it into your virtual environment folder, you can get uninstall it:

sudo apt install python-gobject python-gi-cairo
cp -R /usr/lib/python2.7/dist-packages/gi ~/.virtualenvs/mplpip/lib/python2.7/site-packages/
cp -R /usr/lib/python2.7/dist-packages/glib ~/.virtualenvs/mplpip/lib/python2.7/site-packages/
cp -R /usr/lib/python2.7/dist-packages/gobject ~/.virtualenvs/mplpip/lib/python2.7/site-packages/
cp -R /usr/lib/python2.7/dist-packages/gtk-2.0 ~/.virtualenvs/mplpip/lib/python2.7/site-packages/
cp -R /usr/lib/python2.7/dist-packages/cairo ~/.virtualenvs/mplpip/lib/python2.7/site-packages/
sudo apt purge python-cairo python-gi python-gi-cairo python-gobject python-gobject-2

Launch Jupyter Notebook with `jupyter notebook`. Create a new notebook and check out the available matplotlib plotting backends with %matplotlib -l<?code>; select the one you'd like to use with <code>%matplotlib <backend> before importing matplotlib's pyplot with from matplotlib import pyplot as plt.

matplotlib is very well-documented; check out what's new in the latest release, read more about how to plot and choose a plotting backend, and see examples of figures in the matplotlib gallery.

Caltech has a great intro to Jupyter notebooks, and you can even download their example notebook and play with it yourself.