ArcGIS and Python Virtual Environments part 1

I recently learnt how to get Python virtual environments set up on Windows with ArcGIS installed, and thought it would be useful to write the notes up. This post covers installing a virtual environment.

Why would you want to do this (use virtual environments that is, not use arcpy)?

ArcGIS installs it's own version of python in the c:\python27\ArcGIS10.x folder. Upgrading ArcGIS will cause it's own particular version to be installed in a folder specific to that version. These should be left alone, as upgrading modules in them may cause all sorts of issues and potentially break ArcGIS stuff. So ideally we want to keep the ArcGIS python install as vanilla as possible. But what about when you want to experiment with other libraries, or need to test/develop against a different version of a library?

Virtual environments to the rescue! This post will explain the steps I took to getting Python virtual environments up and running for Windows, with particular notes for those whose Python install is the default ArcGIS installation (which is a little different than normal). However, most of the steps are applicable to any windows Python installation.

I found these sites very handy while reading up on installing virtual environments, and worth referring to if you have problems. In fact, you should open these and have a look through them now.

• https://virtualenv.readthedocs.org/en/latest/ 
• http://docs.python-guide.org/en/latest/dev/virtualenvs/
• http://python-packaging-user-guide.readthedocs.org/en/latest/installing/#optionally-create-a-virtual-environment

There are two main steps.

1. Installing and setting up a Python virtual environment so that we can install additional modules without affecting the ArcGIS python install, and 
2. Getting that virtual environment to interact with the ArcGIS python installation so that we can use arcpy.

To do this we will be working in cmd.exe (the command interpreter/shell/terminal) hereafter called 'cmd'.

This does not normally need to be an administrator level, so open up cmd.

In cmd, navigate to your installed python folder (example is ArcGIS 10.3) by typing:

cd c:\python27\ArcGIS10.3

Next we need to install pip.  Pip is a package management system that can install python packages. If you don't have pp installed, the easiest way to do this for windows is to download get-pip.py from here.

Move the downloaded get-pip.py file to the directory above and in cmd run:

python get-pip.py

Now you should have pip installed, which will make the rest easier.

Add the ArcGIS python folders to the environment path if they are not already. For ArcGIS installs (windows) also add the  \Scripts folder.

The path settings can be found in My Computer > Properties > Advanced System Settings > Environment Variables >
Add the following to the end of the environment path:

     ;C:\Python27\ArcGIS10.3;C:\Python27\ArcGIS10.3\Scripts
 

The next step is to install virtualenv. Virtualenv can be used to separate installs for development purposes. See the Python guide or Kenneth Reitz's virtualenvs github docs.

There is also virtualenvwrapper-win, a wrapper for virtualenv for windows that makes it easier to manage environments. You will need virtualenv installed first though.

To install virtualenv, you should only need to type

pip install virtualenv

In case of error:

Installing modules can be tricky on Windows, in many cases the files will need to be compiled and this might fail if you do not have Visual Studio 2008 installed. There are three potential ways to address this.

1. Install a compiler:
1. Install Visual Studio 2008 Express, or download and install just the compiler tools from https://www.microsoft.com/en-us/download/details.aspx?id=44266 .
2. upgrade setuptools by running pip install --upgrade setuptools
3. open a new cmd prompt and try the install again.
4. type pip install virtualenv
2. Use the  --use-wheel option:
1. Type pip install --use-wheel virtualenv
3. Download the appropriate precompiled wheel file and run directly
    

Install the virtualenv wrapper for windows:

 pip install virtualenvwrapper-win

In order to manage all our potential virtual environments, virtualenvwrapper has some handy features. You can set a default folder for your virtual envs to be installed to!

Type   

export WORKON_HOME=~/Env

to create an "Env" folder in your home directory (In Windows, the default path for WORKON_HOME is %USERPROFILE%Env). This will become the default location. If you don't want to do this, just make sure that you are in the appropriate folder before creating your virtual environment.

Now we are ready to create a virtual environment and switch to it. The following uses virtualenvwrapper. https://virtualenvwrapper.readthedocs.org/en/latest/command_ref.html#managing-environments

In your command prompt type:

mkvirtualenv myenv

then (if needed):

workon myenv

The virtualenv name will appear to the left of the path, e.g.,

           C:\users\~

You can see what is installed by default using pip list to list installed modules. You can now install whatever you need, safe in the knowledge that it will not affect your other install of Python. 

To deactivate the virtual env  and return the to default installed Python, type 

deactivate

The cmd path will now not have the env name next to it, showing that we are out of the virtual environment.

If you installed virtualenvwrapper, you can also type 

rmvirtualenv myenv

to delete it. This will not delete the folder that was made though.
 

In the next post I will cover the setup needed to easily call arcpy from your new virtual environment.