Face Detection

Now that we have an understanding of how and why we do the grayscaling we can now continue to detect face, and eyes using some of the pre-compiled functions in OpenCV, so how do we do that?

Well the nice people at Intel have created a few .xml files containing OpenCV data structures which can be compared to number of scenarios using the CasacadeClassifier function (the definition is available here).

But for the sake of "it" I will quote it here, "The classifier outputs a “1” if the region is likely to show the object (i.e., face/car), and “0” otherwise. To search for the object in the whole image one can move the search window across the image and check every location using the classifier. The classifier is designed so that it can be easily “resized” in order to be able to find the objects of interest at different sizes, which is more efficient than resizing the image itself. So, to find an object of an unknown size in the image the scan procedure should be done several times at different scales.

The word “cascade” in the classifier name means that the resultant classifier consists of several simpler classifiers (stages) that are applied subsequently to a region of interest until at some stage the candidate is rejected or all the stages are passed"

So simple put you get the data from the .xml file, in this case "haarcascade_frontalface_alt.xml" and "haarcascade_eye_tree_eyeglasses.xml"

which can be called using the following method:


String face = "haarcascade_frontalface_alt.xml";
String eyes = "haarcascade_eye_tree_eyeglasses.xml";

So now the program will compare the matrix you gave and try and match the data structures and if they match it will show tell the programmer that yes that is true but in order to make sure that you, the user can see it you need to apply the same method used in highlight different colours (in one of the previous blog entries) to show where the face and eyes may be.

The result:

Note: my colleague's eyes are not highlighted, this is not because of a lack of feature but because of a low grade camera 

Gray Scale and Equalizing Histogram

The short version:

• Here is the function:
• cvtColor(source, gray, CV_BGR2GRAY);\\converts to gray-scale
• equalizeHist( gray , result ); 

• Use the above code and you get a gray-scaled image with an improved contrast range.

The long version:

    The reason behind converting our coloured image to a gray-scaled image is to reduce the size of the matrix the computer has to process in order to deliver the result. An analogy to indulge the reader...

Imagine you have to bottles filled with water or coke (I like coke). One is a gallon bottle and the other is the overpriced 250 ml coke bottle, both have the same bottleneck. Now if you drain them, (into a bucket and a glass respectively never waste coke) the time it takes for the big bottle is incredibly large compared to 250 ml bottle. Similarly when you have a bottleneck such as your processor, you would definitely prefer that it takes less time to process the data you give it and give you a result in as close to real-time as possible.

Now that you have a simpler image you need the aspects of the image to have reasonable variation so that it is easier for the computer to differentiate between two types of grays and therefore we do the contrast magic.

And the result for gray-scaling and contrast enhancement (Histogram Equalization)

Image 1: Grayscale

And,

Image 2: Equalized Histogram

Note that in the second Image 2 the light intensity has been increased and the difference between black and white has been improved.

Colour: Drawing Motion Path and Circle

Once the current location of the object is determined, a path of the motion can be drawn using function Cvline where we need to include the last position of the line and the current position. A circle is drawn by using function cvcircle. The  video was the result from the functions executed.

Flipping Video

When you first use the cvVideoCapture you'd notice that the result doesn't actually mimic your movements (especially if you're using a webcam) the reason is that generally you wouldn't need to use the mirrored results, but when you're testing it is much simpler to have the camera "follow" your movement, in other words mirror your view.

For this I used the flip(source,result,1), where 1 is to flip around the y-axis. Also the both the source and result need to be Mat (matrix) in order to work. The result is below

The word "PROGRESS" is mirrored 

Another Entry 20/02/13

OK, so the problem with developing a object detection software in Visual C++ is that you have to use something like the OPENCV library and the bad thing about it that it is really, really and I mean REALLY a pain to understand what's happening. Probably because I'm a terrible coder/programmer/person but it may as well have to do with the fact that it has little to no documentation as to what the functions that are in there actually do and how and when to implement them. As far as rote learning is concerned you do get a few bits of code here and there but that doesn't really help when you have a dead line and you still haven't managed to DO something about it.

I understand that this post is mostly a rant but I need to rant at the moment.

Back to the problem at hand and the dealing with it bit... The problem at the moment is that I have tried to detect a two shapes (a triangle and a quadrilateral)  but the triangle tends to be detected everywhere (reasons being noise) and the quad isn't detected at all.

Hopefully in a few hours I will have better results as I'm venturing to new territory and re-writing my code.

Colour: Implementing Image Moment for Tracking

Image moment is a weighted average of pixels intensities. Image moment could give useful properties for image analysis like area, centroid and also the image orientaion.
In OpenCV the following function is used to calculate moment;

  CvMoments *moment = (CvMoments*)malloc(sizeof(CvMoments));
        cvMoments(img, moment, 1); 

To get the exact location of the object or the centroid can be obtained by dividing the moment value with the area.The area can be obtained by applying the following function;

  cvGetCentralMoment()

Colour: Reducing Noise

One of the approach to reduce the noise in the image is by using filters. For this project, i've applied Gaussian blur filter to soften the image, eventually helps to reduce the noise. The result is as below;

                                          1) Smoothed image after applying Gaussion blur                                           

                                          2) Non smooth image

Colour: Detecting an object of a particular colour

In OpenCV, images are captured in BGR format. For this project, we convert the format from BGR to HSV as it is most suitable colour space for colour based image segmentation. Each pixel in HSV image is represented by 3 matrices; Hue-represent the color, Saturation-mixture with black, Value-mixture with black. The following functions are used in the code;
-cvCvtcolour(*source, *dest, * code of color conversion )

Next, we threshold the image. This would create a binary image, where the selected colour will turn to white, while other part of the image turns black. The following function is used;
-cvInRangeS(const CvArr* src, CvScalar lower, CvScalar upper, CvArr* dst)

Picture above shows the threshold of the image, in which green colour was selected.

Capturing Video From Webcam

As part of requirement of the project, the program must be able to process a live streaming video. To do that, the program need to request the computer to turn on any available webcam, then show on the screen the video recording. To do that, the following functions are used;

1) cvCaptureFromCAM(0): this function allocates and initialize the cvcapture structure to allow the program read a live streaming video from the webcam. Number (0) represents the index of the camera. If there is more than one camera, the number could be 1,2,3... depending on which camera is going to be used.

2) cvQueryFrame(  ); this function grabs frame from the video sequentially for the processing.

Stage 2

There are few options of ways we can apply to detect an object and then track its movement from a live streaming video. Each method will take a particular variable from the data (sequence of images), process it, and then give result. For this project, we have decided to break it into two parts;

1) the first part uses color as variable to detect an object 

2) use objects edge as variable for the image processing. 

Stage 1

The initial Challenge was deciding what software to use to create the program and the options were either Matlab or Microsoft Visual C++ (VC++). After some discussions with our supervisor, we decided to opt VC++ mainly because both of us know how to code in C++. We're familiar with the syntax and structure. Using matlab would have required gaining understanding of a software that we're not good and that would have wasted time.

The other reason, and this is more important is that Matlab compared to VC++ is painfully slow and has no real world application for Image and Video processing, due it's aforementioned speed issues.

Lastly our Project is build using the OpenCV (Short for: Open Source Computer Vision) library which is "a library of programming functions for real time computer vision." and "is released under a BSD license, it is free for both academic and commercial use. It has C++, C, Python and soon Java interfaces running on Windows, Linux, Android and Mac. The library has >2500 optimized algorithms (see figure below). It is used around the world, has >2.5M downloads and >40K people in the user group. Uses range from interactive art, to mine inspection, stitching maps on the web on through advanced robotics."