Using pyenv to Install a Specific Version of Python
Aug. 27, 2019, noon
GNU/Linux distributions provide recent versions Python as part of the default installation, and in fact, rely on the language to create many of the system tools used by the distribution. For users' Python development needs, the version provided by the distribution may not be appropriate for various reasons. Some distributions -- Fedora, for example, make it easier for users to obtain the version they need by including the Python version as part of the name of the installable package from the distribution's repository. But still the versions available in this way may still not meet the user's needs. And on some distributions such as Arch there is only one cutting edge version of Python available. Installing alternative versions of such software from the AUR will probably cause problems.
An excellent solution to this problem is the pyenv. This utility allows users to install a specific Python variant and version on a per user basis and allows the user to set a specific Python on a per directory basis. This article describes the installation and configuration of pyenv.
Introduction
As the pyenv Github page states:
pyenv lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well. ... pyenv does...In contrast with pythonbrew and pythonz, pyenv does not...
- Let you change the global Python version on a per-user basis.
- Provide support for per-project Python versions.
- Allow you to override the Python version with an environment variable.
- Search commands from multiple versions of Python at a time. This may be helpful to test across Python versions with tox.
- Depend on Python itself. pyenv was made from pure shell scripts. There is no bootstrap problem of Python.
- Need to be loaded into your shell. Instead, pyenv's shim approach works by adding a directory to your $PATH.
- Manage virtualenv. Of course, you can create virtualenv yourself, or pyenv-virtualenv to automate the process.
One element of the pyenv configuration is setting the environment variable PYENV-ROOT to ~/.pyenv. All of its tools and Python versions it installs located in this directory.
The pyenv-root directory.
One element of the pyenv configuration is setting the environment variable
PYENV-ROOT to ~/.pyenv. The screenshot shows the contents of this directory.
Note:The additions to bash configuration files to configure pyenv and pyenv-virtualenv are specific to bash. For the fish sell and zsh, see the respective tools' Github pages, linked in the references section below.
Install and Configure pyenv
Install pyenv
pyenv is available in the repositories of common distributions such as Arch Linux and openSUSE Tumbleweed. It can be installed the default package management tools of the distributions.
On Arch:
[brook:~/DataEXT4/example.com] 4s $ sudo pacman -S pyenv
The above command with its output:
[brook:~/DataEXT4/example.com] 4s $ sudo pacman -S pyenv resolving dependencies... looking for conflicting packages... Package (1) New Version Net Change Download Size community/pyenv 1.2.13-1 2.20 MiB 0.17 MiB Total Download Size: 0.17 MiB Total Installed Size: 2.20 MiB :: Proceed with installation? [Y/n] :: Retrieving packages... pyenv-1.2.13-1-any 171.9 KiB 204K/s 00:01 [##############################################] 100% (1/1) checking keys in keyring [##############################################] 100% (1/1) checking package integrity [##############################################] 100% (1/1) loading package files [##############################################] 100% (1/1) checking for file conflicts [##############################################] 100% (1/1) checking available disk space [##############################################] 100% :: Processing package changes... (1/1) installing pyenv [##############################################] 100% Optional dependencies for pyenv git: installing development versions [installed] :: Running post-transaction hooks... (1/1) Arming ConditionNeedsUpdate...
On openSUSE Tumbleweed:
22:02:46 [brook@G5-openSUSE:~] 4s $ sudo zypper in pyenv
The above command with its output:
22:02:46 [brook@G5-openSUSE:~] 4s $ sudo zypper in pyenv Loading repository data... Reading installed packages... Resolving package dependencies... The following 3 NEW packages are going to be installed: pyenv pyenv-bash-completion pyenv-zsh-completion 3 new packages to install. Overall download size: 256.5 KiB. Already cached: 0 B. After the operation, additional 772.2 KiB will be used. Continue? [y/n/v/...? shows all options] (y): y Retrieving package pyenv-1.2.9-1.4.x86_64 (1/3), 233.1 KiB (771.5 KiB unpacked) Retrieving: pyenv-1.2.9-1.4.x86_64.rpm .................................................................................................................................................................................................[done] Retrieving package pyenv-zsh-completion-1.2.9-1.4.noarch (2/3), 11.7 KiB ( 298 B unpacked) Retrieving: pyenv-zsh-completion-1.2.9-1.4.noarch.rpm ......................................................................................................................................................................[done (9.1 KiB/s)] Retrieving package pyenv-bash-completion-1.2.9-1.4.noarch (3/3), 11.7 KiB ( 406 B unpacked) Retrieving: pyenv-bash-completion-1.2.9-1.4.noarch.rpm .................................................................................................................................................................................[done] Checking for file conflicts: ...........................................................................................................................................................................................................[done] (1/3) Installing: pyenv-1.2.9-1.4.x86_64 ...............................................................................................................................................................................................[done] (2/3) Installing: pyenv-zsh-completion-1.2.9-1.4.noarch ................................................................................................................................................................................[done] (3/3) Installing: pyenv-bash-completion-1.2.9-1.4.noarch ...............................................................................................................................................................................[done]
Configure pyenv
Configuring pyenv is a matter of setting environment variables that pyenv needs to operate. This is done by using export statements in the user's bash at ~/.bash_profile or in certain distributions in the user's bash configuration at ~/.bashrc. Using the echo command can be used to insert the export statements at the end of these files as shown below.
On Arch and openSUSE Tumbleweed:
22:03:18 [brook@G5-openSUSE:~] 15s $ echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
22:06:07 [brook@G5-openSUSE:~] $ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
22:06:38 [brook@G5-openSUSE:~] $ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
The newly set environment variables can be used immediately in the current shell session without restarting the terminal with:
[brook:~/DataEXT4/example.com] $ exec "$SHELL"
pyenv's help lists the available commands.
[brook:~/DataEXT4/example.com] $ pyenv --help
The above command with its output:
[brook:~/DataEXT4/example.com] $ pyenv --help Usage: pyenv[ ] Some useful pyenv commands are: commands List all available pyenv commands local Set or show the local application-specific Python version global Set or show the global Python version shell Set or show the shell-specific Python version install Install a Python version using python-build uninstall Uninstall a specific Python version rehash Rehash pyenv shims (run this after installing executables) version Show the current Python version and its origin versions List all Python versions available to pyenv which Display the full path to an executable whence List all Python versions that contain the given executable See `pyenv help ' for information on a specific command. For full documentation, see: https://github.com/pyenv/pyenv#readme
Installing Specific Python Version
At this point pyenv is installed and configured but the only available Python version is that installed by the distribution. This can be verified with
pyenv versionswhich shows Python versions available to pyenv.
The above command with its output on Arch:
[brook:~/DataEXT4/example.com] $ pyenv versions * system (set by /home/brook/.pyenv/version)
and on openSUSE Tumbleweed:
22:10:02 [brook@G5-openSUSE:~/DataEXT4/example.com] master(+0/-814,7)* ± pyenv versions pyenv: version `3.5.7' is not installed (set by /home/brook/DataEXT4/example.com/.python-version) system
Installing Needed Version
From the output of pyenv versions, above we can see that ony the system Python version is available on both Arch and openSUSE. The openSUSE output is different because the command outputs shown from openSUSE were performed later than the Arch ones and after I had already created a virtualenv based on Python 3.5.7. The virtualenv configuration expected Python 3.5.7, resulting in the error message. Below we install specific versions of Python we need for the project. But first in the two following outputs, we see what happens when we try to install a Python version without specifying the minor version. pyenv informs us available versions that partially match the requested version.
On Arch:
[brook:~/DataEXT4/example.com] $ pyenv install 3.5
The above command with its output:
[brook:~/DataEXT4/example.com] $ pyenv install 3.5 python-build: definition not found: 3.5 The following versions contain `3.5' in the name: 3.3.5 3.5.0 3.5-dev 3.5.1 3.5.2 3.5.3 3.5.4 3.5.5 3.5.6 3.5.7 activepython-3.5.4 pypy3.5-c-jit-latest pypy3.5-5.7-beta-src pypy3.5-5.7-beta pypy3.5-5.7.1-beta-src pypy3.5-5.7.1-beta pypy3.5-5.8.0-src pypy3.5-5.8.0 pypy3.5-5.9.0-src pypy3.5-5.9.0 pypy3.5-5.10.0-src pypy3.5-5.10.0 pypy3.5-5.10.1-src pypy3.5-5.10.1 pypy3.5-6.0.0-src pypy3.5-6.0.0 pypy3.5-7.0.0-src pypy3.5-7.0.0 stackless-3.3.5 stackless-3.5.4 See all available versions with `pyenv install --list'. If the version you need is missing, try upgrading pyenv.
On openSUSE Tumbleweed:
22:10:35 [brook@G5-openSUSE:~/DataEXT4/example.com] master(+0/-814,7)* ± pyenv install 3.5
The above command with its output:
22:10:35 [brook@G5-openSUSE:~/DataEXT4/example.com] master(+0/-814,7)* ± pyenv install 3.5 python-build: definition not found: 3.5 The following versions contain `3.5' in the name: 3.3.5 3.5.0 3.5-dev 3.5.1 3.5.2 3.5.3 3.5.4 3.5.5 3.5.6 activepython-3.5.4 pypy3.5-c-jit-latest pypy3.5-5.7-beta-src pypy3.5-5.7-beta pypy3.5-5.7.1-beta-src pypy3.5-5.7.1-beta pypy3.5-5.8.0-src pypy3.5-5.8.0 pypy3.5-5.9.0-src pypy3.5-5.9.0 pypy3.5-5.10.0-src pypy3.5-5.10.0 pypy3.5-5.10.1-src pypy3.5-5.10.1 pypy3.5-6.0.0-src pypy3.5-6.0.0 stackless-3.3.5 stackless-3.5.4 See all available versions with `pyenv install --list'. If the version you need is missing, try upgrading pyenv.
Note that the available versions of Python 3.5 listed when running the above pyenv command on openSUSE are different than on Arch, meaning I will have to install the common version on Arch later, and use that as the virtualenv version when accessing the project in either Arch or openSUSE.
Below we see an installation specifying the complete Python version.
On Arch:
[brook:~/DataEXT4/example.com] 2 $ pyenv install 3.5.7
The above command with its output:
[brook:~/DataEXT4/example.com] 2 $ pyenv install 3.5.7 [brook:~/DataEXT4/example.com] 2 $ pyenv install 3.5.7 Downloading Python-3.5.7.tar.xz... -> https://www.python.org/ftp/python/3.5.7/Python-3.5.7.tar.xz Installing Python-3.5.7... Installed Python-3.5.7 to /home/brook/.pyenv/versions/3.5.7
On openSUSE:
22:12:40 [brook@G5-openSUSE:~/DataEXT4/example.com] master(+0/-814,7)* ± pyenv install 3.5.6
The above command with its output:
22:12:40 [brook@G5-openSUSE:~/DataEXT4/example.com] master(+0/-814,7)* ± pyenv install 3.5.6 Downloading Python-3.5.6.tar.xz... -> https://www.python.org/ftp/python/3.5.6/Python-3.5.6.tar.xz Installing Python-3.5.6... WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib? WARNING: The Python readline extension was not compiled. Missing the GNU readline lib? WARNING: The Python sqlite3 extension was not compiled. Missing the SQLite3 lib? Installed Python-3.5.6 to /home/brook/.pyenv/versions/3.5.6
The above openSUSE problems can be solved by installing prerequisites:
zypper in zlib-devel bzip2 libbz2-devel libffi-devel libopenssl-devel readline-devel sqlite3 sqlite3-devel xz xz-devel
Install and Configure pyenv-virtualenv Plugin
Install pyenv-virtualenv Plugin
The pyenv-virtualenv plugin for pyenv can optionally be installed to create and manage virtual environments. The installation is a simple cloning of its github repository.
On all distributions
[brook:~/.pyenv] $ git clone --branch v1.1.5 https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
The above command with its output:
[brook:~/.pyenv] $ git clone --branch v1.1.5 https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv Cloning into '/home/brook/.pyenv/plugins/pyenv-virtualenv'... remote: Enumerating objects: 2064, done. remote: Total 2064 (delta 0), reused 0 (delta 0), pack-reused 2064 Receiving objects: 100% (2064/2064), 580.31 KiB | 2.08 MiB/s, done. Resolving deltas: 100% (1413/1413), done. Note: checking out '294f64f76b6b7fbf1a22a4ebba7710faa75c21f7'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b
Configure pyenv-virtualenv Plugin
[brook:~/.pyenv] 127 $ echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
Setting Python Version
Unlike when running pyenv versions previously, we can now see there are other versions available.
[brook:~/DataEXT4/example.com] $ pyenv versions
The above command with its output:
[brook:~/DataEXT4/example.com] $ pyenv versions * system (set by /home/brook/.pyenv/version) 3.5.7
The command pyenv local will display the active Python version for the local directory, if set, or a message to indicating that it is not set.
[brook:~/DataEXT4/example.com] $ pyenv local
The above command with its output:
[brook:~/DataEXT4/example.com] $ pyenv local pyenv: no local version configured for this directory
Setting the version uses the same command but with the desired version specified as below.
[brook:~/DataEXT4/example.com] 1 $ pyenv local 3.5.7
Now pyenv local will show the version set for the local directory.
[brook:~/DataEXT4/example.com] $ pyenv local 3.5.7
Conclusion
In the last section we saw that a version of Python unavailable before installing pyenv and using it to install the desired Python version is set as the Python version for the directory. This Python version will be used in subsequent commands in this directory. It can also be used to create a Python virtual environment using the built in Python 3 method, virtualenv, or pipenv.