Image Processing on the SBC

Having a webcam is all well and good, but what can we do with it? Well as it turns out, rather a lot. Computer vision (CV) is a big part of advanced robotics and CV algorithms have become the subject of much interest in recent years. Allowing your robot to "see" its surroundings is one of the most powerful things you can do to allow it to perform complex tasks. One of the most prominent tools in CV development is OpenCV. OpenCV is a set of libraries developed by Intel to aid in image processing tasks. Luckily for us, OpenCV works well on the SBC and can be installed with minimal difficulty.

Note that many image processing applications are extremely computationally intense and may be too much for the SBC to handle. This makes it important to test your application as early in the development cycle as you can so you can get a good feeling of how the SBC will cope with the amount of work you are requesting of it.

This document is going to walk through setting up OpenCV, and then writing a small program that detects red objects in an image.

Setting up OpenCV

In order to use OpenCV on your machine you will first need to install the libraries and development files. To do this use apt-get install and install the following packages from the Debian package repository:

• libcv2.1
• libhighgui2.1
• libcvaux2.1
• libcv-dev
• libhighgui-dev
• libaux-dev

If apt-get cannot find these packages that may mean there are newer versions available. Use apt-cache search opencv to find out what the most up to date version is and install that instead.

In order to make sure it installed properly, check to see that the files are in the right location.

ls /usr/include/opencv

In particular you are looking for cv.h and highgui.h. Some versions of OpenCV (if you aren't installing 2.1) put highgui.h in a separate folder and give it a slightly different name (highgui_somecharacters.h>. If you do not see both cv.h and highgui.h in /usr/include/opencv then you may have to search around a bit to find them.

Using OpenCV in your program

Once you have installed OpenCV you can begin writing your code. Make sure you include the necessary libraries:

#include <cv.h>
#include <highgui.h>

The first thing we should do is capture an image from the webcam:

CvCapture* capture = cvCaptureFromCAM(CV_CAP_ANY); //select which webcam to capture from (in this case whichever it can find)
if(!capture){ //make sure it found a camera to capture from
	fprintf(stderr, "Error: capture is NULL \n");
	getchar();
	return -1;
}

IplImage* img = cvQueryFrame(capture); //take the picture
IplImage* copy = cvCreateImage(cvGetSize(img),8,3); //create a new image of 8 bit 3 channel.  this will store to modifications we make.

Then we want to check to image and search for red pixels:

for(int i=0;i<(img->height);i++){ //in the image.. go loop until the vertical pixel reaches the height of img
	for(int j=0;j<(img->width);j++){ //in the image.. loop until horizontal pixel reaches the width of img
		s=cvGet2D(img,i,j); //get the RGB values of img's i,j, into a scalar container, s
		if((s.val[2]>atoi(&*argv[1]))&&(s.val[1]<atoi(&*argv[2]))&&(s.val[0]<atoi(&*argv[3]))){ //if(RED>redlower && GREEN<greenupper && BLUE<blueupper)

			c.val[2] = s.val[2];
			c.val[1] = s.val[1];
			c.val[0] = s.val[0];
			cvSet2D(copy,i,j,c);
			redpixelcount++;
		}
		else
		{
			c.val[2] = 255;
			c.val[1] = 255;
			c.val[0] = 255;
			cvSet2D(copy,i,j,c); //make any non-red pixel white
		}
	}
}

Notice we have the limits for color be settable via command-line argument. Once this is all done we can save the images and print out the red pixel tally:

printf("Red pixel count is: %d \n", redpixelcount);

cvSaveImage("original.jpg",img);	
cvSaveImage("redDetect.jpg",copy);

You can download the full source code here.

A note on compiling

To compile you will need to link the correct libraries. To do this use the pkg-config tool. You may need to install it with apt-get before you can use it.

g++ opencvtest.c `pkg-config --cflags --libs opencv`

If you had to search for highgui.h previously because it was not in the same directory as cv.h then you will need to link it separately based on what path it got installed to (likely opencv2 or highgui).

Note that since OpenCV is designed for C++ you will need to use g++ rather than gcc in order to compile.

Results

These tests were run with ./a.out 100 75 75. What that means is it will accept any pixel with an RGB value of >99, <76, <76. These numbers could be fine-tuned a bit so that you cut out the pixels it was picking up in the carpet but it obviously works in principle. The second test even shows that it is sensitive enough to pick up a laser dot from a small laser pointer.