Nicholas Piël

For my Msc thesis I have developed a system build in Python which does person recognition and have shown that it is possible to obtain a better recognition rate with this system than by using Google’s Picasa. I have put the source code online and will hereby announce that I will try my best to spend some time explaining how to do person and face recognition with Python.  I hope that a public announcements such as this will instantly create some public debt forcing me to actually complete this task. We shall see.

The approach described in my thesis uses a combination of pictorial cues (Eigenfaces, SIFT points, color histogram) and contextual cues (co-occurence with other persons or background). The idea behind this is really simple, in order to recognize a person we don’t even need to see the face of that specific person in most cases if we have more contextual information about the setting in which it was taken. For example, lets say you are looking at some pictures from ‘Sesame street’, when you detect Ernie there is a high probability that that other person on that same photo will be Bert. Even more so if we detect that the main component of that other persons color histogram is yellow.

The approach can be divided in three different subtasks:

1. Detecting the person and segmenting its specific region
2. Extracting the features
3. Clustering over these features

In the first task, we will detect the different people by using a face detection technique based on haarcascades. I plan on showing how to use OpenCV with Python and how you can improve its performance by combining the result of multiple haar cascades.

With the detected face we only have a certain square region within a picture which is very likely to contain a face. In order to detect the rest of the body i used a graph based segmentation technique and highly optimized the segmentation algorithm by using implementations in NumPy, Fortran and finally PyCUDA

From this segmented region we will then extract pictorial features such as a color histogram and SIFT features. With this information we can then try to extract and use our contextual information.

In the schematic overview we can see the different steps, we start with some images and segment it in regions of interest. From these regions we will then extract features to build our person models, over which we can then cluster by using  pictorial features (SIFT points and Color Histograms) and contextual features (ie, co-occurence with detected background or other persons). The code for this can already be found on BitBucket but is a bit rough, but as already said I promised to do some explaining. So keep an eye on this blog if you’re interested.

For now, I present you my collected list of references (also in BibTex format) regarding person recognition. It could be a nice starting point for anyone interested in this domain.

References