VirtualEnv and Anaconda – Frustrations
by Gilbert Keith
Having anaconda on your mac is nice (IPython Notebook!)
But it totally breaks things if you are trying to use virtualenv on top of that.
Whenever you create a virtualenv, you get a bad looking error which is not very easy to parse.
GKs-MacBook-Air:~ gk$ virtualenv venv New python executable in venv/bin/python Installing setuptools, pip... Complete output from command /Users/gk/playtestr/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip: Ignoring indexes: https://pypi.python.org/simple/ Exception: Traceback (most recent call last): File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/commands/install.py", line 236, in run session = self._build_session(options) File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/basecommand.py", line 52, in _build_session session = PipSession() File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/download.py", line 216, in __init__ super(PipSession, self).__init__(*args, **kwargs) File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 200, in __init__ self.headers = default_headers() File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/requests/utils.py", line 550, in default_headers 'User-Agent': default_user_agent(), File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/requests/utils.py", line 519, in default_user_agent _implementation = platform.python_implementation() File "/Users/gk/anaconda/lib/python2.7/platform.py", line 1499, in python_implementation return _sys_version() File "/Users/gk/anaconda/lib/python2.7/platform.py", line 1464, in _sys_version repr(sys_version)) ValueError: failed to parse CPython sys.version: '2.7.5 (default, Aug 25 2013, 00:04:04) \n[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]' Storing debug log for failure in /Users/gk/.pip/pip.log ---------------------------------------- ...Installing setuptools, pip...done. Traceback (most recent call last): File "/Users/gk/anaconda/bin/virtualenv", line 11, in sys.exit(main()) File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv.py", line 824, in main symlink=options.symlink) File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv.py", line 992, in create_environment install_wheel(to_install, py_executable, search_dirs) File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv.py", line 960, in install_wheel 'PIP_NO_INDEX': '1' File "/Users/gk/anaconda/lib/python2.7/site-packages/virtualenv.py", line 902, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /Users/gk/venv/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 2
I started off trying to figure out what the hell CPython was, and if I could get rid of it. Turns out, it’s a fool’s errand. Don’t bother with my silly ways.
This is what CPython is (and hopefully you see why messing with it is bad) – From StackOverflow:
So what is CPython
CPython is the original Python implementation. It is the implementation you download from Python.org. People call it CPython to distinguish it from other, later, Python implementations, and to distinguish the implementation of the language engine from the Python programming language itself.
The latter part is where your confusion comes from; you need to keep Python-the-language separate from whatever runs the Python code.
CPython happens to be implemented in C. That is just an implementation detail really. CPython compiles your python code into bytecode (transparently) and interprets that bytecode in a evaluation loop.
What about Jython, etc.
Jython, IronPython and PyPy are the current ‘other’ implementations of the Python programming language; these are implemented in Java, C# and RPython (a subset of Python), respectively. Jython compiles your Python code to Java bytecode, so your Python code can run on the JVM. IronPython lets you run Python on the Microsoft CLR. And PyPy, being implemented in (a subset of) Python, lets you run Python code faster than CPython, which rightly should blow your mind. 🙂
Actually compiling to C
So CPython does itself not translate your Python code to C. It instead runs a interpreter loop. There is a project that does translate Python-ish code to C, and that is called Cython. Cython adds a few extensions to the Python language, and lets you compile your code to C extensions, code that plugs into the CPython interpreter.
Here’s a basic explanation of what’s going on:
- I’m trying to create a virtualenv for an app
- Virtualenv does some calls to underlying libraries for its thing
- There’s something wrong with one of my libraries
After about an hour of searching, I came across this on the python.org bug tracker.
Digging through the whole thing, it became more clear what was going on – Anaconda had introduced something which breaks everything.
The user wesmadrigal had said he’d fixed it:
I just commented out the _sys_version_parser regular expression in anaconda/lib/python2.7/platform.py on line 1363 and replaced it with the _sys_version_parser from /usr/lib/python2.7/platform.py and everything worked fine.
What he’s saying is that if you do this:
GKs-MacBook-Air:~ gk$ python Python 2.7.6 |Anaconda 1.9.1 (x86_64)| (default, Jan 10 2014, 11:23:15) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.__file__ '/Users/gk/anaconda/lib/python2.7/platform.pyc'
and open up the .py file (not the .pyc!) referenced therein, and search for ‘_sys_version_parser’, it should look like this:
_sys_version_parser = re.compile( r'([\w.+]+)\s*' '\(#?([^,]+),\s*([\w ]+),\s*([\w :]+)\)\s*' '\[([^\]]+)\]?')
and not like this:
_sys_version_parser = re.compile( r'([\w.+]+)\s*' '\|[^|]*\|\s*' # version extra '\(#?([^,]+),\s*([\w ]+),\s*([\w :]+)\)\s*' '\[([^\]]+)\]?')
It looks like wesmadrigal has submitted a bug to the folks at Continuum who write code for Anaconda.
So if you run into this error, hopefully the above will fix things.
I don’t know if
conda update conda
conda update anaconda
will break this again, though. Gotta keep an eye out.