Wilson Mar bio photo

Wilson Mar


Calendar YouTube Github


Confusion and errors from too many alternatives and options

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean


This tutorial describes the different options to install and uninstall Python and Python packages within various package managers.

Here I’m taking a “deep dive” approach because I haven’t seen one on the internet.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

I’ve pulled out the various incantations suggested by others on StackOverlow and put them here in context.

TL;DR Summary - It’s a mess


The version of Python that comes with Apple MacOS is obsolete and needs to be updated along with Apple XCode CLI for the MacOS version you’re using.

There are two separate versions of Python: 2 and 3. some Python functions in one version do not work with commands in another version.

This has given rise to several versions of Python frameworks being maintained in parallel. For example, both Django 1.3 and Django 1.0 the web application development framework for Python need to be maintained at the same time.

PROTIP: Various methods of installing Python are incompatible with each other. This has given rise to the need for package managers such as pip (Python Installation Packager) that enable one to switch among different versions of Python installed.

This complexity necessitates the packaging of whole virtual environments to isolate within a folder (directory) everything (all dependencies) that each Python project (application) needs to run. This means duplicated files for each Python application, which consume more disk space.

An additional complication is that there are several alternative virtual environment packagers such as easy_install, virtualenv, and pipenv.

  • MacOS does not come installed with a package manager for Python until pip.
  • pip (Python Installation Packager) is a package manager built on top of setuptools which is what downloads and installs Python packages from the PyPI (Python Package Index) library online at https://pypi.org.

  • Setuptools itself is installed using easy_install.
  • easy_install is an environment manager.

  • Virtualenv (venv) is an environment manager for Python.
  • Conda does both, and is language agnostic (not just for Python).

  • Miniconda is a lightweight distribution of Conda, and uses conda commands.
  • Anaconda is installed on top of miniconda to provide a curated collection of over 720 “common” packages for scientific Python users. It is no longer recommended due to the security vulnerabilities in that many packages. So it’s best to install the base Miniconda and add only the packages you actually use.


Despite all this hassle around versioning, Python is the preferred language of Artificial Intelligence and Machine Learning at the forefront of computer science innovation today. The heavy use of math in AI and ML by TensorFlow means it’s best to install Anaconda and use conda commands (instead of Miniconda or pip with virtualenv).

IPython (Jupyter) Notebook enables a “notebook” interface to re-run commands. See http://sjbyrnes.com/python

PROTIP: PIP install is troublesome, often because they are more recent than those in Conda.

Python Install Options

There is what can be a confusing conflict of choice here for installing Python and its package manager.

easy_install is an environment manager. One writes: “Avoid easy_install or pip to install a Python package that needs a library used by non Python programs, such as Qt bindings (PySide)” or Django*.

PROTIP: If you must use easy_install, delete it after.

Alternatively, use a package manager. CAUTION: MacPorts, Fink, and Homebrew do not coexist well on the same machine.

  • Install using Homebrew, then add homebrew science for scientific work (according to this).

  • MacPorts is an alternative to Homebrew some claim is more compatible with other Linux. However, not all packages are available in MacPorts.

Use Docker

  1. From Dockerhub account “python” get a Docker image on your machine containing Python3 running within Alpine Linux OS:

    docker pull python:3-alpine
  2. Build a container containing a small python environment:

    docker container run -d --memory=20m --memory-swap=20m -name myPython python:3-alpine sleep 3600
  3. Execute the container:

    docker exec -it myPython /bin/sh
  4. Use the container

  5. Stop the Docker instance:

    docker stop python:3-alpine

Obsolete Python comes with MacOS

Ever since the Mavericks version of macOS, Python 2 comes installed on MacOS machines.

(Use the index at the right if you want to jump ahead)

  1. Open a Terminal shell window and issue command:

    python --version

    The response, for example:

    Python 2.7.19

    CAUTION: The sub-version of Python that comes installed with MacOS may be obsolete and needs to be upgraded. But keep to version 2, not version 3 of Python.

  2. Find where Python2 is installed:

    type -a python

    The response:

    python is /usr/bin/python

    You may also see the following if you’ve installed a shim to enable switching of Python versions:

    python is /Users/wilsonmar/.pyenv/shims/python
    python is /usr/bin/python

Folders Python on CLI

  1. List your present working directory:


    We next use a Python interactive command to obtain this again.

  2. Open a Python command-line prompt:


    The response shows that version 2 comes with MacOS:

    Python 2.7.12 (default, Oct 16 2016, 19:01:27) 
    [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.

    The response on a new El Capitan machine:

    Python 2.7.6 (default, Sep  9 2014, 15:04:36)
    [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.

    >>>   is the Python interpreter prompt.

    Don’t enter or copy >>> when typing or copying Python commands.

    Current Working Directory

  3. Display Python’s current working directory:

    >>> import os
    >>> os.getcwd()

    In the response, the “mac” user name is substituted with your user name:


    The above should be the same as the path obtained from pwd before entering Python.

  4. Futhermore…

    >>> import os, sys
    >>> os.path.dirname(os.path.abspath(sys.argv[0]))

    The response is your home folder (substitue “mac” with your user name):


    Exit Python

  5. Exit the Python interpreter by typing the exit function (with the parentheses symbols):

    >>> exit()

    The exit() commmand works the same for both Python 2 and Python 3.

    Using Python 3

  6. After installing Python3, obtain the Python 3 command line with:


    The response I got:

    Python 3.7.5 (default, Nov  1 2019, 02:16:32) 
    [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.

Python 3 vs. 2

Since 2018, many say “all new Python code should be written for version 3. There are so many new features in Python 3 that it doesn’t make much sense to stick with Python 2 unless you’re working with old code.”

Most new features introduced with Python 3 are not backwards compatible with version 2.

Where are Python executables?

  1. Python3 is installed in a different folder than Python 2.

    type -a python3

    The response:

    python3 is /Users/wilsonmar/.pyenv/shims/python3
    python3 is /usr/local/bin/python3
  2. Get the location where Python is installed:

    which python
    which python3

    • Python v2 is installed in /usr/bin/python
    • Python v3 is installed in /usr/local/bin/python3

    If you get /Users/user/.pyenv/shims/python3

  3. Let’s see what’s there:

    ls -al /usr/local/bin/python3 

    The result shows a link to where Homebrew stores:

    lrwxr-xr-x  1 wilson_mar  admin  34 Nov 30 16:27 /usr/local/bin/python3 -> ../Cellar/python/3.7.5/bin/python3

    Python command for Python3

    Some prefer to If you want If you tried to commit suicide like the above, the work-around is an alias, which the operating system resolves before going down PATH.

  4. To use Python3 as the default version for the python command, set in you Mac’s ~/.bash_profile:

    alias python=python3
    alias pip=pip3

For the most part, Python 2 code works with Python 3.

Where Python 2 code fails most often is the print statement. Printing in Python 2 is done like so:

print "Hello", "world!"

The response:

Hello world!

If you input the above in Python 3, the response is:

SyntaxError: Missing parentheses in call to ‘print’

This is because Python 3 uses a function:

print("Hello", "world!")

So in Python 2.6+, use the future module to back-port:

from __future__ import print_function
   print("Hello", "world!")

Try this:

import sys
   print('Python: {}'.format(sys.version))

The response:

Python: 3.5.2 |Anaconda custom (x86_64)| (default, Jul  2 2016, 17:52:12) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)]

Floating point

In Python 2, type in the REPL:


The response is:


In Python3, type:


The response is:


Python programming code

  1. Download or view a Python program:

    By convention, Python programming source files have a file suffix of .py regardless of the version. This is because, unlike on Windows, Linux looks at the first line of script files to identify what program is used to run the file.

    The “shebang” first line in Python 2 programs start with:

    #!/usr/bin/env python

    This is an absolute file system path because the executor doesn’t look on the $PATH for the program defined in ~/.bash_profile.

    The “shebang” first line in Python 3 programs start with:

    #!/usr/bin/env python3

    env is an executable file from virtualenv.

Start a HTTP Server Using Python

A simple HTTP server service can be started with this Python 2 command:

python -m SimpleHTTPServer

For Python3:

python3 -m http.server

The response:

Serving HTTP on port 8000 ( ...   

CAUTION: Hitting Ctrl-C on a Mac, does not shutdown the server gracefully, and the binded address will still be in use.

TODO: Add port designation in command line.

File handling using Pathlib

  1. To create a new folder using Python3: TODO:

  2. To create a new file using Python3, import the pathlib module’s Path object:

    from pathlib import Path

Don’t Uninstall Default Python on macs

The version of Python that comes with macOS should not be removed because some Apple system software have hard-coded references to it.

In this bad advice to harm yourself:

  1. Elevated privilages (sudo) are necessary to remove Python from your Mac

    sudo rm -rf ~/Library/Frameworks/Python.framework/Versions/2.7
    sudo rm -rf "/Applications/Python 2.7"
  2. Remove symbolic links pointing to the python version:

    cd /usr/local/bin/
    ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7' | awk '{print $9}' | tr -d @ | xargs rm
  3. Remove references to deleted paths in PATH environment variable within shell profile files.
    Depending on which shell you use, any of the following files may have been modified:

  4. List symbolic links pointing to the python version:

    ls -l /usr/bin/python

    On El Capitan, this should display: a sym link such as:

    -rwxr-xr-x  2 root  wheel 58416 Jul 14  2015 /usr/bin/python

    Or if instead you followed some bad advice and see something like this:

    lrwxr-xr-x  1 root  wheel  18 Feb  7 20:54 /usr/bin/python -> /usr/bin/python2.7
  5. List symbolic links pointing to the python version:

    cd ~
    ls -l /usr/bin/python/python2.7


    lrwxr-xr-x  1 root  wheel  75 Oct  8 10:46 /usr/bin/python2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7

    The “../../” means that it’s above your HOME folder, in the root of the Mac OS.

  6. So let’s go there:

    cd /System/Library/Frameworks/Python.framework/Versions/2.7/bin/

    There are the executables “python” and “python2.7” plus others.

  7. Run:


    The response:

    Python 2.7.10 (default, Jul 30 2016, 19:40:32) 
    [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
  8. Exit

  9. Run the generic python generically:


    The response is a newer Python:

    Python 2.7.12 (default, Oct 16 2016, 19:01:27) 
    [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> exit()
  10. List symbolic links pointing to the Python version:

    ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7'

    According to this:

Upgrade Python

  1. Install XCode CLI.

Download Python installer

If you must do it the hard way, bareback, etc:

  1. In an internet browser at

  2. Click the Latest link at the top or a
    specific “Mac OS X 64-bit installer” for macOS 10,9+.

    File Date Download
    python-2.7.15-macosx10.9.pkg 2018-05-01 22.7 MB
    python-2.7.12-macosx10.6.pkg 2016-06-25 21.3 MB
    python-2.7.11-macosx10.6.pkg 2015-12-05 21.1 MB
    python-2.7.10-macosx10.6.pkg 2015-05-23 21.1 MB
    python-2.7.09-macosx10.6.pkg 2014-12-10 21.0 MB
    python-2.7.08-macosx10.6.pkg 2014-07-02 19.4 MB
    python-2.7.07-macosx10.6.pkg 2014-05-31 19.4 MB
    python-2.7.06-macosx10.6.pkg 2013-11-10 19.2 MB

    This table shows the growth in download size over time, an analysis unique to this page.

Upgrade pip and setuptools

Many disagree with https://techworm.net/programming/install-pip-python-mac-windows-linux/ which recommends use of easy_install

https://packaging.python.org/discussions/pip-vs-easy-install/ which is part of setuptools.

https://en.wikipedia.org/wiki/Setuptools#EasyInstall easy_install in not a full fledged package installer like pip is.

kinda like a disgraced parent.

Install pip homebrew without setuptools

Instead of following instructions such as this with:

pip install -U pip setuptools

BTW, on Windows it’s:

python -m pip install -U pip setuptools

On Ubuntu 17.04+ Linux:

sudo apt update;

On previous Ubuntu versions, see https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa

sudo add-apt-repository ppa:deadsnakes/ppa
   sudo apt-get-update; sudo apt install python3.x
  1. On a Mac use the Mac system package manager Homebrew to install pip (as recommended by this site and https://pip.readthedocs.io/en/stable/installing):

    brew install pip

    Conda installs outside the standard structure, so

  2. Run:

    brew doctor

    this warning appears (which can be safely ignored):

Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.

Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following "config" scripts:

QUESTION: Is there a way to suppress these messages?


In a GitHub repo cloned locally, if you see a file Requirements.txt, it is likely a list of Python packages needed by the application:

# pip install awscli mkdocs

Notice each specific version is specified.

  1. In a Terminal shell window, change the directory to where app resides.

  2. Just for laughs:

    pip install

    The response:

    You must give at least one requirement to install (see "pip help install")

    The use of the word “requirement” in this message is partly why lists of Python package dependencies are in a requirements.txt file.


  1. Get a Python 2.7 installed. For example, at:


  2. Verify the sym link:

    ls -l /usr/local/bin

    The response:

    lrwxr-xr-x  1 mac  staff  39 Mar  5 00:41 /usr/local/bin/python2.7 -> ../Cellar/python/2.7.12_2/bin/python2.7
  3. Edit ~/.bash_shell to add a shell alias:

    alias python=/usr/local/bin/python2.7
    alias pip=pip3
  4. Close and open another Terminal.
  5. Verify the version.

    cd ~
    python --version

    The response should be the newer sub-version:


Virtual Environments

Examples of instructions for installing a requirements.txt file are typically preceded by a source bin/activate command which executes the activate script in the project’s bin folder.

An activate script is placed in each virtual environment established to store different sets of dependencies required by different projects in separate isolated locaations.

This solves the "Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps your global site-packages directory clean and manageable.

Before the pip install command: Automatically download the packages listed (after you manually change the /path/to)

source bin/activate
   pip install -r /path/to/requirements.txt  

This downloads dependencies from PyPI (the Python Package Index), a public repository of software for the Python programming language at https://pypi.python.org/pypi.

However, adding the “--no-index” option would not use it.

NOTE: pip compiles everything from source if a “wheel” is not available for it. Wheels are a pre-built distribution format that provides faster installation compared to Source Distributions (sdist), especially when a project contains compiled (C language) extensions.

pip install --use-wheel package

pip scikit-learn

  1. In a Terminal on any folder, globally install dependencies libraries:

    pip install -U scikit-learn

  2. Edit the Python script to add at the top:

    from sklearn import tree

    pip iPython Jupyter

  3. iPython is the kernel of Jupyter.

    pip install ipython

Virtualenv and Docker

Hynek Schlawack recommends

  • PROTIP: Don’t pip install anything beyond virtualenv into its global site-packages outside of virtualenv.

  • Install both virtualenv and system isolation (they are not mutually exclusive):

    • Isolate your application server’s OS from its host using Docker/lxc/jails/zones/kvm/VMware/… to one container/vm per application.

    • inside of them also do isolate your Python environment using virtualenv from unexpected surprises in the system site-packages.

Remove hassles from managing per-project virtualenvs by using one of these, depending on the shell and operating system used:

  • virtualenvwrapper
  • virtualenvwrapper-win on MS Windows
  • virtualfish

PIP (Python Installation Packager)

As of Python 2.7.9 and Python 3.4.x, python.org installers for OS X install pip as well from Activestate.com and download ActivePython. It’s a simple install that gives you both Python and pip.

According to https://www.python.org/download/mac/tcltk/, download from http://www.activestate.com/activetcl/downloads file ActiveTcl8.

After install, the ActiveTcl User Guide is popped up.

Jesse Noller notes in So you want to use python on the Mac:

“Now, some people may recommend you install Macports or Fink: these are both “sorta” package managers for OS/X, and while I do have Macports installed, I do not use it for Python work. I prefer compilation and self management.”


Others use easy_install (with setuptools) to install packages from the web.

sudo easy_install pip

The response:

Searching for pip
Best match: pip 8.1.2
Adding pip 8.1.2 to easy-install.pth file
Installing pip script to /Users/mac/miniconda3/bin
Installing pip3.5 script to /Users/mac/miniconda3/bin
Installing pip3 script to /Users/mac/miniconda3/bin
Using /Users/mac/miniconda3/lib/python2.7/site-packages
Processing dependencies for pip
Finished processing dependencies for pip

Virtual pip environments

The best way to have painless and reproducible deployments is to package whole virtual environments of the application you want to deploy including all dependencies but without configuration.

In the world of Python, an environment is a folder (directory) containing everything that a Python project (application) needs to run in an organised, isolated fashion.

When it is initiated, it automatically comes with its own Python interpreter

  • a copy of the one used to create it - alongside its very own pip.

The ability to work with either version 3 or 2.7 on the same machine is needed because, as this MacWorld article points out, Mac Mavericks and Yosemite are installed with Python 2.7, cannot run python3 scripts.

You can work on a Python project which requires Django 1.3
while also maintaining a project which requires Django 1.0.

It’s done by creating isolated Python environments using virtualenv (Virtual python environment builder).

sudo pip install virtualenv

As the reponse requests, activate:

source /usr/local/opt/autoenv/activate.sh

This does not issue a response.

  1. Instead of "venv", substitute your project name to to create:

    cd my_project_folder
    virtualenv venv

    Exclude the virtual environment folder from source control by adding it to the git ignore list.

    New python executable in /Users/mac/gits/wilsonmar/shippable/one/bin/python
    Installing setuptools, pip, wheel...
      Complete output from command /Users/mac/gits/wils...pable/one/bin/python - setuptools pip wheel:
      Traceback (most recent call last):
      File "<stdin>", line 4, in <module>
      File "/Users/mac/miniconda3/lib/python2.7/tempfile.py", line 32, in <module>
     import io as _io
      File "/Users/mac/miniconda3/lib/python2.7/io.py", line 51, in <module>
     import _io
    ImportError: dlopen(/Users/mac/gits/wilsonmar/shippable/one/lib/python2.7/lib-dynload/_io.so, 2): Symbol not found: __PyCodecInfo_GetIncrementalDecoder
      Referenced from: /Users/mac/gits/wilsonmar/shippable/one/lib/python2.7/lib-dynload/_io.so
      Expected in: flat namespace
     in /Users/mac/gits/wilsonmar/shippable/one/lib/python2.7/lib-dynload/_io.so
    ...Installing setuptools, pip, wheel...done.
    Traceback (most recent call last):
      File "/Users/mac/miniconda3/bin/virtualenv", line 11, in <module>
      File "/Users/mac/miniconda3/lib/python2.7/site-packages/virtualenv.py", line 711, in main
      File "/Users/mac/miniconda3/lib/python2.7/site-packages/virtualenv.py", line 944, in create_environment
      File "/Users/mac/miniconda3/lib/python2.7/site-packages/virtualenv.py", line 900, in install_wheel
     call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
      File "/Users/mac/miniconda3/lib/python2.7/site-packages/virtualenv.py", line 795, in call_subprocess
     % (cmd_desc, proc.returncode))
    OSError: Command /Users/mac/gits/wils...pable/one/bin/python - setuptools pip wheel failed with error code 1

    QUESTION: How to fix this? This occurs when virtualenv was installed with easy_install (or “python setup.py install”)

  2. List all virtual environments:


  3. To use a particular Python interpreter:

    virtualenv -p /usr/bin/python2.7 venv

  4. Activate your project:

    source venv/bin/activate

    The name of the current virtual environment should now appear on the left of the prompt (e.g. (venv)Your-Computer:your_project UserName$).

    From now on, any package that you install using pip will be placed in the venv folder, isolated from the global Python installation.


  5. To automatically activate an environment when you cd into it:

    brew install autoenv

    The response:

    ==> Downloading https://github.com/kennethreitz/autoenv/archive/v0.1.0.tar.gz
    ==> Downloading from https://codeload.github.com/kennethreitz/autoenv/tar.gz/v0.
    ######################################################################## 100.0%
    ==> Caveats
    To finish the installation, source activate.sh in your shell:
      source /usr/local/opt/autoenv/activate.sh
    ==> Summary
    🍺  /usr/local/Cellar/autoenv/0.1.0: 4 files, 5K, built in 2 seconds
  6. Install packages as usual, for example:

    pip install request

  7. When you are done working in the virtual environment for the moment:


    The above puts you back to the system’s default Python interpreter with all its installed libraries.

  8. To delete a virtual environment, delete its folder. In this case, it would be:

    rm -rf venv

  9. To keep your environment consistent,


    the current state of the environment packages:

    pip freeze > requirements.txt

    This creates (or overwrites) a requirements.txt file containing a simple list of all the packages in the current environment and their respective versions. This file would make it easier to re-create the environment and to install the same packages using the same versions:

    pip install -r requirements.txt

    This ensures consistency across installations, deployments, and developers.

As noted in http://docs.python-guide.org/en/latest/dev/virtualenvs/ this create a folder in the current directory containing the Python executable files, and a copy of the pip library which you can use to install other packages. The name of the virtual environment (in this case, it was venv) can be anything; omitting the name will place the files in the current directory instead.

  1. Open an internet browser to https://www.python.org/downloads/ and download file python-3.4.2-macosx10.6.pkg.

    See http://docs.python-guide.org/en/latest/dev/virtualenvs/ https://www.digitalocean.com/community/tutorials/common-python-tools-using-virtualenv-installing-with-pip-and-managing-packages

  2. To list what packages have been installed:

    pip list

  3. Look for packages by keyword:

    pip search django

Use pip to install Tensorflow

As Siraj shows in his video, on a Mac Terminal, define an environment variable that points to the download URL:

export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-0.10.0rc0-py3-none-any.whl

(This is for Python 3 on a Mac.)

Install it using PIP and the variable:

sudo pip3 install --upgrade $TF_BINARY_URL


This explanation forks another.

Setup a provider VM solution to store the image (like VMware, AWS, Hyper-V). https://www.virtualbox.org/wiki/Downloads click “x86/amd64” next to VirtualBox 4.3.20 for OS X hosts to download VirtualBox- (109 MB) Read the 368 page User Manual.

Install https://github.com/dotless-de/vagrant-vbguest to keep VirtualBox guest additions up to date. http://schof.org/2014/03/31/working-around-virtualbox-bug-12879/

In the Finder, in the Applications folder, drag and drop it onto your Dock for quicker use later. Double-click on VirtualBox for the VirtualBox Manager.

Download the 224.3 MB vagrant_1.7.1.dmg

The binary gets installed in the Applications folder with a link to the /usr/bin so it is added to the shell path.

List commands:

vagrant list-commands

Change directory to where you want to store the Vagrant project and run

vagrant init

The response:

A Vagrantfile has been placed in this directory. You are now ready to vagrant up your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on vagrantup.com for more information on using Vagrant.


This shows Vagrantfile (with capital V).


vagrant up --provider=PROVIDER

https://vagrantcloud.com/boxes/search lists boxes created by the community.

I am pulling the box from ATT M2X https://m2x.att.com/developer/sample-code This Repo provides a Vagrant virtual machine that contains several demo applications (Ruby and Python) that report data to AT&T M2X. https://github.com/attm2x/m2x-demo-vagrant

git clone https://github.com/attm2x/m2x-demo-vagrant.git

vagrant box add chef/centos-6.5


User data for Vagrant is filed in the directory from which vagrant was used and is stored in an invisible directory .vagrant.d

Python 2 executables

This advice from 2010.

Path to executables

  1. To see what MacOS

    which python

    If you have MiniConda installed:


    If you have Anaconda installed and your user is “mac”:


    If you are inside a conda activated environment:

  2. For a list of what Python executes:

    ls ~/miniconda3/bin

    The response begins with this:

    2to3        openssl        python2-config
    activate    pip         python2.7
    aws         pip2        python2.7-config

    Type python

  3. Find where you are picking up Python from?

    type python

    If Python was installed:

    python is hashed (/usr/bin/python)

    Alternately, if Conda was installed:

    python is hashed (/Users/mac/miniconda3/bin/python)

  4. Open a Terminal shell window and issue command:

    python --version

    The response is its version. My Mac Yosemite default of Python shows this:

    Python 2.7.6 (default, Sep  9 2014, 15:04:36)
    [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.

    With Miniconda installed on El Capitan, it’s this instead:

    Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:43:17) 
    [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    Anaconda is brought to you by Continuum Analytics.
    Please check out: http://continuum.io/thanks and https://anaconda.org
  5. Get the folder:

    python -m site --user-site

The response I got is this:




Anaconda Install

PROTIP: Skip to <a href=#Miniconda-install”>install Miniconda</a> instead because there is so much in Anaconda that there is inevitably vulnerabilities lurking there.

  1. Go to web page:


    QUESTION: Is there a brew anaconda?

  2. Click on the operating system icon (Mac, Windows, Linux) or scroll down and press the tab.
  3. Click to download the “command-line installer”.

    | Version    | File                          | Installer | Installed |
    | :--------- | :---------------------------- | -------: | ----: |
    | Python 3.6 | Anaconda3-4.3.0-MacOSX-x86_64 | 363 MB | - |
    | Python 2.7 | Anaconda2-4.3.0-MacOSX-x86_64 | 358 MB | 1.41 GB |
  4. In a (bash) Terminal:

    cd Downloads
    chmod 555 ./Anaconda3-4.3.0-MacOSX-x86_64.sh 

    The response:

    Welcome to Anaconda3 4.3.0 (by Continuum Analytics, Inc.)
    In order to continue the installation process, please review the license
    Please, press ENTER to continue
  5. Press Enter (as if you cared).
  6. Press Tab until you’re exhausted.
  7. Type yes and press Enter.

    Anaconda3 will now be installed into this location: /Users/mac/anaconda3

  8. Press ENTER to confirm the location
  9. Press CTRL-C to abort the installation
  10. Or specify a different location below

  11. Press Enter.


  12. Wait for it to come back to you.

    installing: _license-1.1-py36_1 ...
    installing: alabaster-0.7.9-py36_0 ...
    installing: anaconda-client-1.6.0-py36_0 ...
    installing: anaconda-navigator-1.4.3-py36_0 ...
    installing: appnope-0.1.0-py36_0 ...
    installing: appscript-1.0.1-py36_0 ...
    installing: astroid-1.4.9-py36_0 ...
    installing: astropy-1.3-np111py36_0 ...
    installing: babel-2.3.4-py36_0 ...
    installation finished.
    Do you wish the installer to prepend the Anaconda3 install location
    to PATH in your /Users/mac/.bash_profile ? [yes|no]
    [yes] >>> yes
  13. Type yes.

    You may wish to edit your .bashrc or prepend the Anaconda3 install location:
    export PATH=/Users/mac/anaconda3/bin:$PATH
    Thank you for installing Anaconda3!
    Share your notebooks and packages on Anaconda Cloud!
    Sign up for free: https://anaconda.org

Miniconda install

  1. Automated install from https://github.com/wilsonmar/mac-setup/blob/master/mac-setup.zsh

  2. Setup run-time Environment variables: https://github.com/wilsonmar/mac-setup/blob/master/.zshrc

Below is a more “hands-on” description than what pydata.org and Kyle Purdon offers.

  1. Use an internet browser to visit the Miniconda Download page at http://conda.pydata.org/miniconda.html

    Alternately, download:

    cd ~/Downloads
    wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh

  2. Select version to download. For various versions of Python:

    Version File Size
    Python 3.10 Miniconda3-latest-MacOSX-x86_64.sh 53.9 MB
    Python 3.9 Miniconda3-latest-MacOSX-x86_64.sh 67.8 MB
    Python 3.5 Miniconda3-latest-MacOSX-x86_64.sh 23.4 MB
    Python 2.7 miniconda3-latest-MacOSX-x86_64.sh 20.3 MB

    NOTE: Python3 is not backward compatible with Version 2.

    Notice the “.sh” means these are shell scripts.

  3. Open a Terminal shell window to navigate to your Downloads folder and run the Python 2.7 script:

    cd ~/Downloads
    bash miniconda3-latest-MacOSX-x86_64.sh -b

    PROTIP: The “-b” option specifies unattended with defaults.

    The response:

    Welcome to miniconda3 4.0.5 (by Continuum Analytics, Inc.)
    In order to continue the installation process, please review the license
    Please, press ENTER to continue
  4. Press Enter to accept the license.
  5. Press “q” to the “:” prompt.
  6. Type yes.

    “ERROR: File or directory already exists:” appears if miniconda was already installed.

  7. Confirm conda version.

Miniconda Uninstall

  1. Install a package to uninstall itself:

    conda install anaconda-clean
    anaconda-clean --yes

    find / -name anaconda3 2>/dev/null

  2. Delete the anaconda install folder, which is usually under your home dir:

    rm -rf /usr/local/anaconda2
    rm -rf /usr/local/anaconda3

Conda Uninstall

  1. Install a package to uninstall itself:

    conda install anaconda-clean
    anaconda-clean --yes

    find / -name anaconda3 2>/dev/null

  2. Delete the anaconda install folder, which is usually under your home dir:

    rm -rf /usr/local/anaconda2
    rm -rf /usr/local/anaconda3

Conda version

See https://uoa-eresearch.github.io/eresearch-cookbook/recipe/2014/11/20/conda/

  1. For the version of conda installed, specify the upper-case V:

    conda -V


    conda --version

    The response is like:

    conda 4.11.0

Update conda

  1. To update miniconda’s version, use the conda command line installed above:

    conda update conda

    The response is a list of packages to be updated if you agree:

    Fetching package metadata: ....
    .Solving package specifications: ..........
    Package plan for installation in environment /Users/mac/miniconda3:
    The following packages will be downloaded:
     package                    |            build
     sqlite-3.13.0              |                0         1.4 MB
     python-2.7.12              |                1         9.5 MB
     conda-env-2.5.1            |           py27_0          26 KB
     ruamel_yaml-0.11.7         |           py27_0         174 KB
     conda-4.1.8                |           py27_0         201 KB
                                            Total:        11.2 MB
    The following NEW packages will be INSTALLED:
     ruamel_yaml: 0.11.7-py27_0
    The following packages will be UPDATED:
     conda:       4.0.4-py27_0 --> 4.1.8-py27_0 
     conda-env:   2.4.5-py27_0 --> 2.5.1-py27_0 
     python:      2.7.11-0     --> 2.7.12-1     
     sqlite:      3.9.2-0      --> 3.13.0-0     
    Proceed ([y]/n)? 
  2. Press “y” to proceed. A sample response:

    Fetching packages ...
    sqlite-3.13.0- 100% |################################| Time: 0:00:02 483.10 kB/s
    python-2.7.12- 100% |################################| Time: 0:00:20 477.84 kB/s
    conda-env-2.5. 100% |################################| Time: 0:00:00 392.63 kB/s
    ruamel_yaml-0. 100% |################################| Time: 0:00:00 331.95 kB/s
    conda-4.1.8-py 100% |################################| Time: 0:00:00 457.60 kB/s
    Extracting packages ...
    [      COMPLETE      ]|###################################################| 100%
    Unlinking packages ...
    [      COMPLETE      ]|###################################################| 100%
    Linking packages ...
    [      COMPLETE      ]|###################################################| 100%
  3. After install, close and then re-open the terminal window so the changes can take effect.

  4. For a list of packages installed locally (in the currently active environment):

    conda list

    The “py36_1” in the list are pip installed.

    _license                  1.1                      py36_1  
    alabaster                 0.7.9                    py36_0  
    anaconda                  4.3.0               np111py36_0  

.bash_profile config

TODO: Instructions for both Miniconda and conda.

The path to Python should be the first in PATH:

  1. Open a text editor to ~/.bash_profile and add:

    export PATH=”~/miniconda3/bin:$PATH”
    export PYTHON_PATH=~/miniconda3/bin/python

    When you’re done:

  2. Activate the shell file :

    source ~/.bash_profile

    Troubleshoot brew install

  3. To reset:

    brew link --overwrite python
    Warning: Already linked: /usr/local/Cellar/python/2.7.12_2
    To relink: brew unlink python && brew link python

Scripting Python versions

TODO: Use a shim so that a script can display the version of Python it is using:

Python 3.6.7 (default, Nov 20 2018, 22:03:43) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.44.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.version)  # parentheses necessary in python 3.

Some answers:

3.6.7 (default, Nov 20 2018, 22:03:43) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.44.4)]
2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)]

Per https://docs.python.org/3/library/sys.html#sys.hexversion

>>> sys.version_info
sys.version_info(major=3, minor=6, micro=7, releaselevel='final', serial=0)
(2, 5, 2, 'final', 0)
>>> sys.hexversion
>>> '%x' % sys.hexversion
>>> sys.hexversion < 0x02060000

For a script to ensure that it’s running the version of Python intended:

import sys
if sys.version_info<(2,6,0):
  sys.stderr.write("You need python 2.6 or later to run this script\n")


>>> from platform import python_version
>>> print(python_version())
import platform
from distutils.version import StrictVersion 
assert(StrictVersion(platform.python_version()) >= "2.6")

Conda info

  1. Get a list:

    conda info

    The response:

    Current conda install:
              platform : osx-64
           conda version : 4.3.9
        conda is private : False
       conda-env version : 4.3.9
     conda-build version : 2.1.2
          python version : 3.5.2.final.0
        requests version : 2.12.4
        root environment : /Users/mac/anaconda  (writable)
     default environment : /Users/mac/anaconda
        envs directories : /Users/mac/anaconda/envs
           package cache : /Users/mac/anaconda/pkgs
            channel URLs : https://repo.continuum.io/pkgs/free/osx-64
             config file : None
            offline mode : False
              user-agent : conda/4.3.9 requests/2.12.4 CPython/3.5.2 Darwin/16.4.0 OSX/10.12.3
                 UID:GID : 501:20

    Conda environments

  2. Get a list of Conda environments (from any folder) using the -e flag:

    conda info -e

    Alternately, if you like typing long options:

    conda info --env

    The response are like this:

    tensorflow_env           /Users/mac/anaconda/envs/tensorflow_env
    root                  *  /Users/mac/anaconda
  3. Create a Conda environment named py2 using Python 2.7:

    conda create -n py2 python=2.7 anaconda
  4. Press y to go ahead.

  5. Add python packages, such as TensorFlow:

    conda install -n yourenvname tensorflowp
    conda install -c conda-forge tensorflow
  6. Activate to use the environment:

    source activate
  7. When done using TensorFlow, deactivate the environment:

    source deactivate

Conda pyenv

pyenv enables switch between multiple versions of Python on a single system (laptop).

  1. Create two “named” Conda environments (one with Python2 and the other with Python3):

    conda create -n py3 python=3*
    conda create -n py2 python=2*
  2. Set one of these as my default by adding to my terminal startup file ~/.bash_profile:

    source activate py3 

    Typically I only use these “named python” environments to run a Python REPL or do general Python tasks. I’ll create another conda environment named specifically for each real project I work on.


PROTIP: Delete Conda one folder at a time (without the –yes parameter).


Files such as:

   Backup directory: /Users/mac/.anaconda_backup/2017-03-07T051620


Install Python packages

From inside a conda environment:

  1. NumPy at http://www.numpy.org needed by http://www.pymvpa.org/installation.html

    pip install numpy

  2. Instead of downloading http://www.scipy.org/scipylib/download.html# linear algebra, standard distributions, signal processing, data IO

    pip install scipy

  3. SKlearn

    pip install sklearn

  4. Pandas based on VIDEO: How to install Pandas Miniconda:

    pip install pandas

    YouTube videos: Learn Pandas

  5. matplotlib

    Other Python packages:

    • xlwings interfaces with Microsoft Excel spreadsheets
    • pygame develops GUI

OpenCV for computer vision

Follow the instructions below to install python2 + OpenCV in mac


In case of python3 + OpenCV follow


Conda vs Pip

  • Conda handles library dependencies outside of the Python packages as well as Python packages themselves.

  • Conda installs from binary, meaning that someone (e.g., Continuum) has already done the hard work of compiling the package, making installation easier, and faster.

  • pip can install anything from PyPI in one command.
  • Conda requires at least three commands: skeleton, build, install, and possibly more.

  • Conda uses its own format, which has some advantages (like being static, and again, Python agnostic).

Conda provides an alternative set of commands popular for scientific (Machine Learning) computing. See http://conda.pydata.org/docs/_downloads/conda-cheatsheet.pdf

This table lists the difference in commands between Conda and pip, a summary of the more detailed table is at https://conda.io/projects/conda/en/latest/commands.html#conda-vs-pip-vs-virtualenv-commands

Task Conda package and environment manager command Pip package manager command Virtualenv environment manager command
Install a package conda install $PACKAGE_NAME pip install $PACKAGE_NAME -
Update a package conda update --name $ENVIRONMENT_NAME $PACKAGE_NAME pip install --upgrade $PACKAGE_NAME -
Update package manager conda update conda Linux/OSX: pip install -U pip Win: python -m pip install -U pip -
Uninstall a package conda remove --name $ENVIRONMENT_NAME $PACKAGE_NAME pip uninstall $PACKAGE_NAME -
Create an environment conda create --name $ENVIRONMENT_NAME python - cd $ENV_BASE_DIR; virtualenv $ENVIRONMENT_NAME
Activate an environment source activate $ENVIRONMENT_NAME - source $ENV_BASE_DIR/$ENVIRONMENT_NAME
Deactivate an environment source deactivate - deactivate
Search available packages conda search $SEARCH_TERM pip search $SEARCH_TERM -
Install package from specific source conda install --channel $URL $PACKAGE_NAME pip install --index-url $URL $PACKAGE_NAME -
List installed packages conda list --name $ENVIRONMENT_NAME pip list -
Create requirements file conda list --export pip freeze -
List all environments conda info --envs - Install virtualenv wrapper,
then lsvirtualenv
Install other package manager conda install pip pip install conda -
Install Python conda install python=x.x - -
Update Python conda update python * - -

PyCharm IDE

$195/year PyCharm is purpose-built by Jetbrains.com for Python coding:

  1. VIDEO: Read:


  2. Install using Homebrew:

    brew install pycharm
  3. Add Terraform and HCL plugin ( Windows)

    To run Terraform from IDE and autocompletion, rename refactoring, etc.

  4. Configure in Settings -> Tools -> Terraform

    Explicitly add Terraform run configurations


  5. Run Terraform from IDE File context menu -> Run…

    Alternately, click green ‘run’ next to resources

Turi (Dato) Python algorithms

GraphLab Create from Dato provides scalable “pre-implemented” ML algorithms using Anaconda. Entire courses on its use is at

  • https://www.coursera.org/learn/ml-foundations
  • https://www.turi.com/learn/userguide/
  • https://www.turi.com/products/create/docs/
  • https://github.com/learnml/machine-learning-specialization
  • https://www.coursera.org/learn/ml-clustering-and-retrieval/supplement/iF7Ji/software-tools-you-ll-need-for-this-course

When the one-year free license is over, note scikit-learn also uses Python with Anaconda.

Python libraries

For matrix operations, use the Numpy open-source Python library for fast performance with data that fits in memory. Quickstart.

In a requirements.txt file:

tweepy (http://www.tweepy.org)

csv (https://pypi.python.org/pypi/csv)

textblob (https://textblob.readthedocs.io/en/dev/)

keras (https://keras.io)

bokeh Flask ipython jupyter matplotlib nose numpy pandas Pillow pymc requests scikit-image scikit-learn scipy seaborn statsmodel tensorflow virtualenv virtualenvwrapper


brew tap homebrew/science && brew update
brew install opencv3 --with-contrib --with-python3 --without-python
ln -s /usr/local/opt/opencv3/lib/python3.6/site-packages/cv2.cpython-36m-darwin.so /usr/local/lib/python3.6/site-packages/cv2.so
export PYTHONPATH=$PYTHONPATH:/usr/local/Cellar/opencv3/3.2.0/lib/python3.6/site-packages

Data Manipulation

SFrame is an open-source, highly-scalable Python library for data manipulation. Unlike Pandas, SFrame is not limited to datasets which can fit in memory, so it can deal with large datasets, even on a laptop.



Conda is similar to virtualenv and pyenv, other popular environment managers.




conda install numpy pandas matplotlib

conda install jupyter notebook

  1. List the packages installed, with its version number and what version of Python:

    conda list

Conda Environments

  1. Create new environment for Python, specifying packages needed:

    conda create -n my_env python=3 numpy pandas

  2. Enter an environment on Mac:

    source activate my_env

    On Windows:

    activate my_env

    When you’re in the environment, the environment’s name appears in the prompt:

    (my_env) ~ $.

  3. Leave the environment

    source deactivate

    On Windows, it’s just deactivate.

  4. Get back in again.

  5. Create an environment file by piping the output from an export:

    conda env export > some_env.yaml

    When sharing your code on GitHub, it’s good practice to make an environment file and include it in the repository. This will make it easier for people to install all the dependencies for your code. I also usually include a pip requirements.txt file using pip freeze (learn more here) for people not using conda.

  6. Load an environment metadata file:

    conda env create -f some_env.yaml

  7. List environments created on your machine:

    conda env list

  8. Remove an environment:

    conda env remove -n some_env


  • https://conda.io/docs/using/index.html

  • https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/

Where installed?

When Python is installed using pip, see where it’s installed:

python3 -m site --user-base

The response on my system:




https://gist.github.com/alyssaq/f60393545173379e0f3f describes install of https://bootstrap.pypa.io/get-pip.py

https://docs.python.org/3/library/2to3.html 2to3 is a Python program that reads Python 2.x source code and applies a series of fixers to transform it into valid Python 3.x code.


https://joernhees.de/blog/2014/02/25/scientific-python-on-mac-os-x-10-9-with-homebrew/ recommends several pip libraries

https://linuxacademy.com/cp/socialize/index/type/community_post/id/14209 released 1/25/2017 by Michael Jenkins makes use of AWS Boto3 with Python2 to create S3 buckets, upload files, and delete S3 buckets in code examples as part of Linux Academy’s Red Hat Certified Specialist in Virtualization (EX318) Preparation video Course

Sikuli Python

  1. To use SikuliX from real Python via py4j

    cd /usr/local/
    git clone https://github.com/RaiMan/sikulix4python
    cd sikulix4python
  2. Get support on the Python:

    pip install py4j
    Collecting py4j
      Downloading https://files.pythonhosted.org/packages/04/de/2d314a921ef4c20b283e1de94e0780273678caac901564df06b948e4ba9b/py4j- (196kB)
      |████████████████████████████████| 204kB 402kB/s 
    Installing collected packages: py4j
    Successfully installed py4j-
  3. Download the Jython interpreter for Python scripting (the default) jython-standalone-2.7.1.jar

    curl -O https://repo1.maven.org/maven2/org/python/jython-standalone/2.7.1/jython-standalone-2.7.1.jar

  4. Download the sikulixapi for use by Maven:





VIDEO COURSE: EC-Council’s CodeRed “Black Hat Python: Python For Pentesters” by Cody Jackson

More on Python

This is one of a series about Python:

  1. Python install on MacOS
  2. Python install on MacOS using Pyenv
  3. Python install on Raspberry Pi for IoT

  4. Python tutorials
  5. Python Examples
  6. Python coding notes
  7. Pulumi controls cloud using Python, etc.
  8. Jupyter Notebooks provide commentary to Python

  9. Python certifications

  10. Test Python using Pytest BDD Selenium framework
  11. Test Python using Robot testing framework
  12. Testing AI uses Python code

  13. Microsoft Azure Machine Learning makes use of Python

  14. Python REST API programming using the Flask library
  15. Python coding for AWS Lambda Serverless programming
  16. Streamlit visualization framework powered by Python
  17. Web scraping using Scrapy, powered by Python
  18. Neo4j graph databases accessed from Python