I can haz SciPy !!1!1!

"The Dependency Hell", rightmost panel of Hieronymus Bosch's "The Garden of Earthly Delights".
Well, at least it looks like so (the self-test does not smoke anymore).

I will try to reconstruct a walkthrough, but bear in mind that I have installed, uninstalled and reinstalled so much stuff in my Mac that the reproducibility of this recipe should be taken with a grain of salt. If you try it and it works, I would be thankful if you’d leave a note describing exactly what you did.

Without further ado, how to install the trio NumPy, SciPy and Matplotlib in Mac OS X Snow Leopard, using MacPorts:

  1. Download Apple’s Xcode for Mac OS X, without which nothing else is possible. You’ll have to register on Apple, but there is a free inscription if you are not planning to sell anything;
  2. Download MacPorts. This is the package manager which will operate all the magic — it will do for your Mac the same thing yum or apt-get does for a Linux box, so if it doesn’t work properly nothing else will;
  3. Install Xcode and then MacPorts. If you want to be extra sure, follow the intructions on the Macports page — basically instructing you to install X11 support on Mac and MacPorts. I didn’t do this (because I only became aware of those instruction very late in the whole process), but that doesn’t mean you shouldn’t. Alternatively, if you already have MacPorts installed, ensure it is up-to-date by typing:

    sudo port selfupdate

  4. You can compile your dependencies with any compiler — in theory. I found out that SciPy smokes unless compiled with GCC 4.4, but now I have some dependencies compiled with their default choice and some (NumPy, SciPy) in which I forced compilation with GCC 4.4. If you want to try your luck, start by downloading GCC 4.4 and making it default:

    sudo port install gcc44
    sudo port install gcc_select
    sudo gcc_select mp-gcc44

    If instead, you want to reproduce exactly my crazy (but successful) sequence, install GCC 4.4 but do not make it default (omit the last two lines), and also install GCC 4.5 (I know, how many versions of GCC does one need ?!):

    sudo port install gcc45

  5. Install the “non-dependencies” of PIL. Those “non-dependencies” are libraries needed by other libraries in other to provided optional (but often important) functionality, which won’t be installed by the package manager otherwise (PIL itself is a “non-dependency” of SciPy):

    sudo port install jpeg libpng tiff lcms freefont-ttf

  6. Install PIL. This will install python as a dependency. (I have chosen to install 2.7, so I’ve consistenly chosen py27-* packages through the process):

    sudo port install py27-pil

  7. Install the binary dependencies of NumPy and SciPy:

    sudo port install arpack
    sudo port install SuiteSparse

  8. Finally, install the goodies:

    sudo port install py27-numpy
    sudo port install py27-scipy

    Again, if you want to do exactly what I did, the process is more convoluted. After typing exactly those commands above, I then uninstalled then (but not their dependencies):

    sudo port uninstall py27-scipy py27-matplotlib

    And reinstalled everything again (but the dependencies) with the compiler directive:

    sudo port install py27-numpy configure.compiler=macports-gcc-4.5
    sudo port install py27-numpy configure.compiler=macports-gcc-4.4

    I don’t remember why I chose GCC 4.5 for NumPy and 4.4 for SciPy — but my history file is a more faithful testimony than my memory (probably I just mistyped NumPy’s and meant 4.4 for both — give-me some slack, it was almost 4 a.m. by then !).

  9. Last, but not least, install Matplotlib:

    sudo port install py27-matplotlib

I don’t know if all this mix-and-match of compilers is a good thing — all I know is that SciPy smokes if not compiled with GCC 4.4, but maybe compiling everything with GCC 4.4 is the way to go.

To test the installation, call python and selftest the packages one by one:

import numpy

import scipy

import matplotlib

I’ve got a pristine regression test for NumPy and SciPy, including the problematic C++ code weaving. My Matplotlib regression smoked, with several “ImageComparisonFailure: images not close” errors with RMS values slightly above acceptable. I compared the images by hand, however, and could not spot the difference — and dismissed the problem as a matter of different engines of font smoothing.

By the way, if you want to make the new python default, there is a python_select for this job:

sudo port install python_select
sudo python_select python27

Both gcc_select and python_select allow you to choose betwen a list of options which include the MacPorts and the Apple versions of GCC and python, to see the options available, type;

gcc_select -l
python_select -l

And a little something I found out at my expense. When using bash, sometimes updating the PATH is not enough: recently used commands will be at a hash table, which will have to be refreshed, lest bash will still be fetching the command at the wrong path. To check and clear this hash table you can use, respectively, the commands:

hash -r

Well, it is not the Seventh Circle of Heaven, but it is half-way through the Purgatory. Not bad for this Dante, who’s got only Google and StackOverflow for Virgile !

5 thoughts on “I can haz SciPy !!1!1!

  1. All this, or you could just try to get a Matlab Academic license…

    I understand the difficulties of buying software within the academic world in Brazil. Yet, getting someone with your skills dealing with such issues to get software installed is equivalent of getting a highly educated surgeon cleaning the bedsheets at a hospital and thinking the end-result is “saving money”.

    • You are right, of course, but buying a package like MATLAB or Mathematica is often not the end of the story: you still have to build a infrastructure to make those work (they have a cluster backbone, which I don’t know how complex is to manage), you have to learn to work with them, you have to acquire the specialist packages for signal processing, and then you have to maintain all the infra-structure working — so in terms of TCO it is more than just getting a licensing.

      (Of couse, much probably it is still better than all this awful complication.)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s