Wednesday, December 24, 2008

Computer Vision on OS X with Python and OpenCV

After my macbook's sudden demise and spontaneous, inexplicable regeneration, I've decided to try and port some things I've worked on in GNU/Linux to OS X. Also, having a built-in microphone and camera on a portable computer is pretty amazing when you work primarily with audio and video and need to test things.

I was really happy with how my previous forays in pygame had been going, so over the holidays I thought I 'd try and make a version of my previous camLoops prototype, but for OS X.
Also, at Alexandre Quessy's invitation, I've added cam_loop.py to the ToonLoop project. More on that in the coming weeks, so stay "tooned" (if not put off by that terrible pun).

*UPDATE The initial opencv version of camLoops is now available. I've only tested it on a MacBook running OS X so far, but it should also work in GNU/Linux.

I did some research online and there did not seem to be much in the way of camera- frame-grabbing modules for OS X with python bindings (granted it is a pretty niche area). The most complete are Apple's QTKit, the Cocoa framework (which has python bindings) for Quicktime and OpenCV, or Open Source Computer Vision, which is a cross-platform, BSD-licensed library written in C with Python bindings. Since I have no interest in wasting time using a Mac-only, proprietary framework like QTKit, the choice was pretty obvious. The installation, however, was not.

Given my heavy use of fink packages (when I should really just give in and roll GNU/Linux on my laptop), I am used to having to fight with cross-platform librairies/frameworks. I was even pleased to find an OS X specific build instruction page on the OpenCV wiki. It's quite likely that my approach is not the best solution for building but it's what worked for me.

I decided for better or worse to get the current production version of Python, which is 2.6.1 at the time of this writing. I grabbed the disk image and ran the installer (and breathed a sigh of relief). For pygame to work, I had to get PyObjC, which is used to build Cocoa apps for OS X in Python. This is not required for OpenCV, so skip ahead if you're not using pygame. This also required an svn checkout of the 1.4 branch of pyobjc which works on Tiger, as no slick disk-images with installers are available from the website at present. To grab the branch, do:

svn co http://svn.red-bean.com/pyobjc/branches/pyobjc-1.4-branch pyobjc

Fortunately, the PyObjC people know their target audience and all it took once in the pyobjc directory to make a nice installer was:

python setup.py bdist_mpkg --open


*UPDATE Previously OpenCV was using CVS for version control, they have since migrated to Subversion. Check out a clean version with:

svn co https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary opencvlibrary
In the INSTALL file, it is suggested that you do autoreconf -i --force
This failed for me (even though my autotools are up to date) so i used the pre-existing configuration files and left autoconf alone.


In the opencv directory, create a build directory and enter it with mkdir build; cd build


From the build directory, run configure with a few flags set (replace sw with /opt/local if you are using darwinports instead of fink):

../configure CPPFLAGS="-I/sw/include" LDFLAGS="-L/sw/lib" --with-python

Now compile and install with make && sudo make install. You will be prompted for your password.

Edit your ~/.profile to include the following two lines, which will be different if you set the --prefix option on your configure script to something other than the default /usr:

LD_LIBRARY_PATH=/usr/local/lib:${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH
PYTHONPATH=/usr/local/lib/python2.6/site-packages:${PYTHONPATH}
export PYTHONPATH

To test that everything worked, run your python interpreter and try import opencv. Provided that there is no errors such as 'no module opencv', you can start trying the python examples in the samples directory.


I ran python facedetect.py 0, where 0 is the index of the camera you want the program to use. The program then draws a red square around what it thinks is your face. Extra points for being beard proof, and hats off to Mark Asbach for his work on OS X support for OpenCV.

7 comments:

Alexandre Quessy said...

Here is a quote from the Mac_OS_X_OpenCV_Port
page on the OpenCV wiki : Unfortunately, we still have no working (i.e. native) video writer for Mac OS X. If you need video writing, you should currently configure for ffmpeg or xine and disable QuickTime. The downside of this is that you will lose QuickTime video input at the same time. And still ffmpeg video writing support is as buggy on Mac OS X as it is on Linux (for OpenCV). Volunteers?

...that means it is not possible to save movies, unless doing a hack. That would involve copying the frames in Python memory and saving it in a movie using the ffmpeg Python bindings. The pyffmpeg project doesn't look too much active, though.

Tristan Matthews said...

On the same page, in the Task List, under the heading "3. Video I/O", it seems like they've implemented support for uncompressed frames written to a MOV container. There's also an example of writing frames to avi file at http://opencv.willowgarage.com/wiki/Mac_OS_X_OpenCV_Port?action=AttachFile&do=view&target=export.cpp
I haven't played around with this yet but it would be where I would start.
In other news, pygame now has MIDI functionality (via pyportmidi) in the svn!

Tristan Matthews said...

sorry that link got truncated, export.cpp is under the heading Other Resources on that same wiki page, right below the Task List.

Alexandre Quessy said...

Hi !
Yeah, actually, it would be better to use uncompressed video, since it would be very easy to save as a file, and then reload it to start over from where you left it. An other solution would be to use the cpickle module - and maybe an other application to convert that cpickle file to an actual movie - but it might be slower that way.

To be continued.

Meanwhile, I have embedded toonloop.py into twisted, and added some command line options. See http://code.google.com/p/toonloop/

Jinho Yoo said...

Hello, I made another manual to build python binding of OpenCV in Mac OS X.

Unknown said...

Hello, I would like to learn about computer vision could you advise me some books? I would like to work with python and pygame. At home i have and o'reilly book: "Learning OpenCV
Computer Vision with the OpenCV Library" which doesn't teach much about computer vision it focus more about teaching us how to use the OpenCV library using C. Thank you for the help, regards joão

Unknown said...

Hey, I'm a bit late to the party and things have matured a bit. But thanks for the post. I'm just starting to teach myself OpenCV to do some facial recognition stuff just as a proof of concept. The info here helped a bit. Thanks :)