Virtual environments and dependency management in Python

Piotr Grzesik

What I do ?

What I will be talking about ?

  • How to manage multiple python versions ?
  • How to manage virtual environments ?
  • How to manage project dependencies ?

Pyenv - Python version management


  • Tool for managing multiple versions of Python interpreter
  • Supports CPython, PyPy, Stackless, JPython, IronPython
  • Similar to nvm (node.js) and rvm, rbenv (Ruby)
  • Does not work on Windows

How pyenv works ?

Pyenv inserts ~/.pyenv/shims at the beginning of PATH, which contains shims, small executables, which are responsible for passing commands to correct pyenv.


When we're invoking a command e.g. pip, following steps are performed:

  1. Search PATH for executable called pip
  2. Find pyenv shim named pip
  3. Execute shim named pip, which passes command to correct pyenv

How pyenv chooses Python version ?

  1. Environment variable PYENV_VERSION (pyenv shell command)
  2. .python-version file in current directory (pyenv local command)
  3. .python-version file in parent directories (search up to /)
  4. ~/.pyenv/version file (pyenv global command)

Pyenv update


  • Pyenv plugin that provides option to update pyenv with pyenv update command
  • Updates not only pyenv, but also other installed plugins, e.g. pyenv-virtualenv
  • Updates available Python versions

How to use pyenv ?

pyenv install 3.6.2
pyenv install anaconda3-4.4.0 

pyenv versions

pyenv uninstall 2.7

Managing available Python versions

pyenv version

pyenv shell 3.6.2
pyenv local anaconda3-4.4.0 
pyenv global 3.6.2

Managing currently used Python version

Virtual environments in Python

  • Independent, isolated environments
  • Each environment has separate dependencies (e.g. two environments with different versions of the same library)
  • Allow testing and using third-party libraries without polluting global Python installation
  • Part of standard library starting from Python 3.3 (PEP 405)

How virtualenv works ?

  • Activate - inserts /path/to/env/bin at the beginning of PATH
  • sys.path - list of search paths for modules during imports
  • 3rd party virtualenv - modified, orig-prefix.txt allows to include standard library (sys.prefix)
  • venv - pyvenv.cfg file (sys.prefix i sys.base_prefix)
  • Packages are installed to /path/to/env/lib/pythonX.Y/site-packages
home = /usr/bin
include-system-site-packages = false
version = 3.5.2

Example pyvenv.cfg file:

How to use virtualenvs ?

pip install virtualenv
virtualenv /path/to/env

Python < 3.3

pyvenv /path/to/env

Python 3.3 and higher (deprecated since version 3.6)

Recommended since version 3.5

python3 -m venv /path/to/env

Activate newly created virtualenv

source /path/to/env/bin/activate

Deactivate virtualenv


Pyenv + virtualenv


  • Pyenv plugin for managing virtual environments with pyenv
  • Has support for managing conda environments
  • Supports third-party virtualenv as well as venv from standard library

How to use pyenv-virtualenv ?

pyenv virtualenv 3.6.2 my-venv
pyenv virtualenv my-other-venv

Creating virtualenvs with pyenv-virtualenv

pyenv activate my-venv
pyenv deactivate

Working with virtualenvs

Listing existing virtualenvs

pyenv virtualenvs

Removing virtualenvs

pyenv virtualenv-delete my-venv



  • Handles python versions, environments and packages management
  • Multi-platform (Win/macOS/Linux)
  • Supports pip for installing packages in conda environments
  • Installs binary packages (no need to compile during installation)
  • Possible to install packages written in other languages like C/C++, Java, Scala, JavaScript, FORTRAN, R
  • Comes with Anaconda and Miniconda

How to use conda ?

conda create --name my-env
conda create --name my-other-env keras
conda create --name my-another-env python=3.6
conda create --name cloned-env --clone my-env

Creating environments

activate my-env
source activate my-env
source deactivate

Working with environments

Listing existing environments

conda info --envs
conda env list

Removing environments

conda remove --name my-env --all

How to use conda ?

conda install scipy
conda install scipy=0.12.0
conda install --name my-env scipy

Installing packages

conda list

List installed packages

Uninstalling packages

conda remove scipy
conda remove --name my-env scipy

Why should we care about dependency management ?

  • List of external libraries that our application/script needs to run (very often in form of requirements.txt file)
  • Logical separation of dependencies (e.g. separate list for tests, separate list for people to read, separate list for machines)
  • Stable and repeatable builds on different environments



Generates requirements.txt file with dependencies, based on imported packages in your project


Example usage:


Example result in requirements.txt:

pipreqs /path/to/project



  • Two separate lists of dependencies - for humans and for machines
  • Tool that let's you "pin" your dependencies
  • pip-compile + pip-sync

How to use pip-tools ?

requests file:

pip-compile --output-file requirements.txt

Using pip-compile to produce requirements.txt:

requirements.txt file:

certifi==2017.7.27.1      # via requests
chardet==3.0.4            # via requests
click==6.7                # via flask
idna==2.6                 # via requests
itsdangerous==0.24        # via flask
jinja2==2.9.6             # via flask
markupsafe==1.0           # via jinja2
urllib3==1.22             # via requests
werkzeug==0.12.2          # via flask

How to use pip-tools ?

pip-compile --upgrade-package flask

Update specific package:

pip-compile --generate-hashes

Using pip-compile with generating hashes:

requirements.txt file with hashes:

certifi==2017.7.27.1 \
    --hash=sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5 \
    --hash=sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704 \
    # via requests

Syncing current environment with requirements.txt:

pip-sync requirements.txt



  • "Requirements.txt" 2.0 - proposition of new standard
  • Supported by PyPA (Python Packaging Authority)
  • Replace "requirements.txt" with Pipfile and Pipfile.lock
  • Pipfile - uses TOML syntax, contains top-level dependencies
  • Pipfile.lock - contains deterministic set of dependencies, used for creating repeatable environments for your application

Pipfile example

url = ""
verify_ssl = true

pytest = "*"

requests = "*"

python_version = "3.6"

Pipfile.lock example

    "_meta": {
        "hash": {
            "sha256": "a97c6ee0bb0a72c606f78b9f7a8088b61d838cb6312a26574d2126ad09acfead"
        "host-environment-markers": {
            "implementation_name": "cpython",
            "implementation_version": "3.6.2",
            "os_name": "posix",
            "platform_machine": "x86_64",
            "platform_python_implementation": "CPython",
            "platform_release": "4.4.0-53-generic",
            "platform_system": "Linux",
            "platform_version": "#74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016",
            "python_full_version": "3.6.2",
            "python_version": "3.6",
            "sys_platform": "linux"
        "pipfile-spec": 3,
        "requires": {
            "python_version": "3.6"
        "sources": [
                "url": "",
                "verify_ssl": true
    "default": {
        "certifi": {
            "hashes": [
            "version": "==2017.7.27.1"



  • Tool recommended by PyPA
  • Mixture of pip, Pipfile, virtualenv
  • Automatically creates virtualenvs for you
  • Automatically adds/removes dependencies from Pipfile during installation/uninstallation
  • Creates Pipfile.lock file
  • Can install dependencies from Pipfile.lock
  • By default uses hashes for packages verification
  • Integrates with pyenv for python version management

How to use pipenv ?

pipenv --python 3.6.2

Creating new project:

pipenv install requests
pipenv install pytest --dev
pipenv uninstall requests

Installing and uninstalling packages:

Installing all dependencies:

pipenv install
pipenv install --dev

Spawning a shell with virtualenv:

pipenv shell

How to use pipenv ?

pipenv graph

Showing dependency graph:

pipenv check

Checking installed dependencies for security vulnerabilities:

Generate lockfile:

pipenv lock


Installing packages: pip, conda, pipenv

Managing Python installations: conda, pyenv

Environments: virtualenv, pyenv-virtualenv, conda, pipenv

Managing dependencies: pipreqs, pip-tools, pipenv



Virtual environments and dependency management in Python

By progressive

Virtual environments and dependency management in Python

  • 638
Loading comments...

More from progressive