Introduction to Image Processing¶
Image processing is a large and important topic that is all about using computers to process and manipulate images. Here are just a few of the problems that image processing deals with:
- Converting a color image to black and white or grayscale. For example, newspapers that printed in black and white must convert color images to black and white.
- Applying special effects to images, such as sharpening, blurring, scaling, etc.
- Removing “noise” from images. For example, little “specks” and visual imperfections occasionally appear in pictures, and it is often possible to automatically remove them.
- Recognizing image features, such as faces or text. For instance, many digital cameras have facial recognition software that puts a box around faces.
- Efficiently compressing images into small files so that they can be more easily processed. This is especially important in video begin sent over a network, where you might have to send dozens of individual pictures (i.e. frames of the video) a second.
Image processing applications can be computationally intensive because typically every pixel of an image must be processed. For example, a 640x400 pixel image has 256,000 pixels, and if you wanted to, say, convert each pixel to black and white, then you must change all 256,000 pixels. If you are processing an image in real time (e.g. the output of a webcam), then things are even tougher because you might have to process, say, 24 images per second (which amounts to changing more then 6 million pixels each second). Clearly, speed is important in real-time image processing.
In these notes we’ll see a few simple yet useful applications of image processing in Processing. We’ll use this image:
These notes are in part based on Intro to Image Processing notes on the Processing website.
Displaying an Image¶
First, lets see how to display a single image:
PImage img;
void setup() {
img = loadImage("kids.jpg");
size(img.width, img.height);
}
void draw() {
background(0);
image(img, 0, 0);
}
Recall that images are placed according to their upper-left corner.
Playing with tint¶
The easiest way to process an entire image in processing is to use the
tint function:
PImage img;
void setup() {
img = loadImage("kids.jpg");
size(img.width, img.height);
}
void draw() {
background(0);
float g = map(mouseY, 0, 500, 0, 255);
tint(g);
image(img, 0, 0);
}
Here we pass one number, g, to tint that controls the image’s
brightness. If you pass in a second value, you can control the transparency:
PImage img;
PImage bg;
void setup() {
img = loadImage("kids.jpg");
bg = loadImage("bg.jpg");
size(bg.width, bg.height);
}
void draw() {
background(bg);
float g = map(mouseY, 0, 500, 0, 255);
tint(255, g);
image(img, 0, 0);
}
By passing in three parameters, you can control the amount of red, green, and blue for each pixel:
PImage img;
void setup() {
img = loadImage("kids.jpg");
size(img.width, img.height);
}
void draw() {
tint(255, 0, 0); // just red
//tint(0, 255, 0); // just green
//tint(0, 0, 255); // just blue
//tint(100, 10, 230); // a little bit of red, a tiny bit of
// green, and a lot of blue
image(img, 0, 0);
}
How to Access Image Pixels in Processing¶
The tint function is useful, but many image processing algorithms need to
work at the pixel level. That is, they need to scan through an image
one pixel at a time, changing the pixels as they go.
For example, one way to convert a color image to black and white is to look at every pixel and change it to white if it is bright enough, and to black otherwise.
Every PImage object contains an array of pixels that you can access. An
array is like an ArrayList, but simpler and with more compact notation for
accessing its elements.
For example, suppose you have a tiny 3x3 image that we’ll call img:
a b c
d e f
g h i
This little image contains exactly 9 pixel. The pixels are stored in an array like this:
0 1 2 3 4 5 6 7 8
--- --- --- --- --- --- --- --- ---
img.pixels | a | b | c | d | e | f | g | h | i |
--- --- --- --- --- --- --- --- ---
The individuals pixels are accessed using square-bracket notation, e.g.
img.pixels[0] is pixel a, img.pixels[1] is pixel b, and so on.
To use img.pixels in your programs you must call loadPixels and
updatePixels:
img.loadPixels();
// ... code that processes img.pixels
img.updatePixels();
For example, this code converts a color image to grayscale:
img.loadPixels();
int i = 0;
while (i < img.pixels.length) {
color c = img.pixels[i];
float r = red(c);
float g = green(c);
float b = blue(c);
float gray = (r + g + b) / 3;
img.pixels[i] = color(gray); // set the pixel to a shade of gray
// proportional to the brightness
// of the color
++i;
}
img.updatePixels();
The resulting image now consists of 256 different shades of gray, and so is ready to be printed on a black-and-white printer.
Processing Every Pixel of an Image¶
Here’s a complete program that is useful for experimenting with pixel- processing:
PImage img;
void setup() {
img = loadImage("kids.jpg");
size(img.width, img.height);
println(img.width * img.height + " pixels");
}
void draw() {
image(img, 0, 0);
}
void mousePressed() {
println("mouse pressed");
img.loadPixels();
int i = 0;
while (i < img.pixels.length) {
color c = img.pixels[i];
float r = red(c);
float g = green(c);
float b = blue(c);
// the code below is the only code you should change
float gray = (r + g + b) / 3;
img.pixels[i] = color(gray); // set the pixel's new color
// end of code to change
++i;
}
img.updatePixels();
}
As it stands, the program converts img to grayscale when you click the
mouse. Here are a few other ideas:
Show only a single color component of each pixel, e.g.:
img.pixels[i] = color(r, 0, 0);
Re-arrange the color components, e.g.:
img.pixels[i] = color(g, b, r);
“Invert” the components, e.g.:
img.pixels[i] = color(255 - r, 255 - g, 255 - b);
Make pure black and white, e.g.:
float br = (r + g + b) / 3.0; // approximate brightness if (br < 128) { img.pixels[i] = color(0); } else { img.pixels[i] = color(255); }You can use colors other than black and white if you like, e.g.:
float br = (r + g + b) / 3.0; // approximate brightness if (br < 128) { img.pixels[i] = color(255, 255, 0); } else { img.pixels[i] = color(244, 238, 224); }
Alternate Program¶
The following is alternative to the image-processing program above with these added features:
- Clicking the left mouse-button flips back and forth between the original image and the processed image.
- While in the processed image, moving the mouse pointer will cause the image to be
immediately re-drawn. Thus and processing that takes into account, say, the position
of the mouse pointer (e.g. the
blackAndWhitefunction) will be re-processed immediately.
PImage img; // the image to display
PImage orig; // the original image
PImage proc; // the processed image
boolean showProcessed;
void setup() {
// load the original image
orig = loadImage("kids.jpg");
// make a blank image the same size ad the original
proc = createImage(orig.width, orig.height, RGB);
// show the unprocessed image initially
img = orig;
showProcessed = false;
// set the window to be the exact size of the image
size(img.width, img.height);
println(img.width + " x " + img.height + " = "
+ img.width * img.height + " pixels");
}
void draw() {
image(img, 0, 0);
}
void mouseMoved() {
if (showProcessed) {
println(mouseY);
processImage();
img = proc;
}
}
void mousePressed() {
showProcessed = !showProcessed;
if (showProcessed) {
processImage();
img = proc;
}
else {
img = orig;
}
}
void processImage() {
proc.loadPixels();
int i = 0; // for half image use: proc.pixels.length / 2
while (i < orig.pixels.length) {
color c = orig.pixels[i];
proc.pixels[i] = blackAndWhite(c);
++i;
}
proc.updatePixels();
}
color grayscale(color c) {
float r = red(c);
float g = green(c);
float b = blue(c);
float brightness = (r + g + b) / 3;
return color(brightness);
}
color blackAndWhite(color c) {
float r = red(c);
float g = green(c);
float b = blue(c);
float brightness = (r + g + b) / 3;
// set the threshold to be proportional to mouseY
float pct = map(mouseY, 0, height, 0.0, 1.0);
float threshold = 255 * pct;
if (brightness < threshold) {
return color(0);
} else {
return color(255);
}
}