Rudi-Ellen Final Project

This final project is based on the interaction between a captured live video and two LEDs.

The reference was the BrightnessThresholding sample code in the examples of  Processing. What I want to do is to let the LEDs blink according to where a small dot travels inside the Processing screen. To realize this goal, I need to do the following things:

  1. capture the live video with a camera;
  2. let the video be black and white so that I can easily divide different areas of the taken screen;
  3. create a dot that travels inside the video;
  4. let the dot change its color every time it travels to a different area;
  5. let the red LED light when the dot travels to black areas; let the green LED light when the dot travels to white areas.

At first, I was thinking about using a class to create a function of the traveling dot. However, after writing the code out, I found that the color of the dot cannot change according to different colors of the area it’s in because the variable c(color) in the fill() function is not a global one — it is in the class dot(). The function cannot understand it. So I tried to delete the class part of my code and directly describe how the dot travels and changes its color in the void draw() part.

This is the code of Processing:

import processing.serial.*;

Serial myPort;
int valueFromArduino;

int c;
int xpos = width/2;
int ypos = height/2;
int xspeed = 10;
int yspeed = 10;

color black = color(0);
color white = color(255);
int numPixels;
Capture video;

void setup() {
size(640, 480);
video = new Capture(this, width, height);


numPixels = video.width * video.height;
myPort = new Serial(this, Serial.list()[1], 9600);

void draw() {
if (video.available()) {;
int threshold = 127;
float pixelBrightness;
for (int i = 0; i < numPixels; i++) {
pixelBrightness = brightness(video.pixels[i]);
if (pixelBrightness > threshold) {
pixels[i] = white;
else {
pixels[i] = black;

int testValue = get(xpos, ypos);
float testBrightness = brightness(testValue);
if (testBrightness > threshold) {
c = 0;
else {
c = 255;

xpos = xpos + xspeed;
ypos = ypos + yspeed;
if (xpos > width) {
xspeed = xspeed * -1;
if (xpos < 0) {
xspeed = xspeed * -1;
if (ypos > height) {
yspeed = yspeed * -1;
if (ypos < 0) {
yspeed = yspeed * -1;

This is the code of Arduino:

int valueFromProcessing;
int ledPinG = 12;
int ledPinR = 13;

void setup() {
pinMode(ledPinG, OUTPUT);
pinMode(ledPinR, OUTPUT);

void loop() {
while (Serial.available()) {
valueFromProcessing =;

if (valueFromProcessing == ‘H’) {
digitalWrite(ledPinG, HIGH);
digitalWrite(ledPinR, LOW);
} else {
digitalWrite(ledPinG, LOW);
digitalWrite(ledPinR, HIGH);

This is the final effect:


Leave a Reply