Interaction Lab Midterm Project: Magic Glove

Author: Wanchen Zhao (Leslie)
Instructor: Antonius
Project: Magic Glove
Partner: Frank
Materials: solderless breadboard, Arduino, wires, 10k resistors, flex sensors, FSR sensors, glove, Processing

Inspiration & Description
We got our inspiration from the Marvel movie Dr.Strange, where Dr. Stephen Strange releases different patterns combining basic geometric patterns and mysterious texts made up of Greek letters. We decided to recreate this effect using Arduino and Processing. Since we can’t do Dr.Strange’s tempting magic, we hide the trigger to the magic into a glove, that can generate likewise patterns that we drew with Processing code from scratch with a simple movement.

The Process
Drawing with Processing

The visual effect is the most time-consuming and most challenging part of our project. We each drew one pattern with Processing. I started off with the outline structure, or the basic patterns of ellipses and rectangles. To better the effect, I proceeded to adding Greek letters into spaces between the outer circles. The Greek letters come from the internet, all I had to do was copying and pasting them into the “text()” code. But a very time-consuming process was to find a font that supports Greek letters. Luis introduced a website called dafont.com to me, where I could download free fonts, put them into my code’s folder and run it. The function installed in Processing is “PFont myFont” in set up, “myFont = createFont(“AppleMyungjo,50))”, where “AppleMyungjo” is the font and 50 is the size. Not a lot of fonts support Greek letters, and at first for quite a few times I ended up with the “α”s and “β”s unrecognized. I realized it would be so much easier to use English letters, but since we were inspired by Dr.Strange, it was best to make the replication as close as possible. And in order to make the letters go around the circle, I also used the “rotate(radians(counter))” function.
This is how my final animated effect looks like:


The change of color is achieved by a special function in “stroke()” or “fill()”, called “map()”. By inserting “frameCount()” into the parenthesis, Processing enables the pattern to change colors and brightness gradually, giving the pattern a glowing effect.

Serial Communication

Whenever the flex sensor is pressed, the screen generates a pattern. We’re going to hide the two flex sensors in two gloves, so that they each controls different patterns. This is how the flex sensors and resistors are connected to the breadboard:
IMG_7778

WechatIMG226
Before enabling Arduino to talk to Processing, this is to show that the flex sensor is able to send signals when pressed:


Below is the effect of connecting the Arduino with Processing. The way the code works is to print “1” when one flex sensor senses the pressure, and print “2” when another senses the pressure. When both are pressed or neither are pressed, it prints out “0”. Afterwards all we had to do is to use Serial communication, and assign “1” and “2” to the two patterns.

A Sensor Change

However, turns out at the end of the day, our biggest obstacle was not drawing the patterns with Processing; it was the sensor. The flex sensors refused to work anymore right at the night before the presentation, so we had to switch sensors at the last minute. I thought about my Stupid Pet Trick project, where I used FSR to replicate a smart pillow that can sense a person’s head on it. I recalled that FSR is much more sensitive than flex sensors, so we borrowed the last two left in the equipment room. Since I was familiar with FSR because of my Stupid Pet Trick, I did the physical installation and the Arduino code, without any staff help. Fortunately, the Processing code requires almost no change besides the set up.

Other Challenges

Actually, both of the FSR sensors didn’t come in handy at first as well. One had broken wires and need to be re-soldered; the other was the one I used for my Stupid Pet Trick, which had no wires. For the latter we tried using clips along with M/M wires, but the wires fell off easily and the wires still weren’t long enough to be attached to the gloves. Our final solution was to re-solder both. Jiwon helped us with the soldering. It took a while but it was worth it, because our wires never fell off from the breadboard again afterwards.

IMG_7788

IMG_7787

When we ran it afterwards, there was something very weird with my code: no matter how I rewrote the code, there was always an extra set of Greek letters at the bottom right corner of my circle; what’s more, it was always green, refusing to change color and brightness along with the rest of the pattern. When everything else was done, during the debugging process, Jiwon helped us check it and it turns out the problem came from a repetition of the “textAlign”.

After attaching the FSRs to the gloves, we made a edited video, sound effect and subtitles attached:

The Final Versions of the Codes

 

At the source code section, both the Processing code and the Arduino code can be found.

Gains from the Project

Taking Interaction Lab is the first time I encounter computer programming, and combining it with physical interaction. Even though during the first half of the semester we practiced every Friday during lab, it’s not constant enough for me to truly deepen my knowledge and understanding of Processing and Arduino. For midterm project, since we’re drawing the Dr.Strange patterns completely with Processing, I can say I have truly obtained the basic Processing drawing and animation skills we have discussed so far in class. What’s more, with digging into the Processing tutorials and the kind help from our fellows, I have also explored more functions that have polished my drawing, such as RGB, PFont, and text(). I don’t major in IMA, and before taking Interaction Lab I have always thought I can never be able do anything cool on computer. It’s fascinating to discover that I actually have potential in this field. What’s more, I also familiarized myself with Arduino’s sensor installation and Serial communication,enabling myself to transfer signals between Arduino and Processing.

Second, as a student of finance major, most of the academics I deal with have rules and fixed answers, so it’s refreshing and inspiring to encounter IMA, where creativity is just as important as rules and knowledges. I used to think coding was fixed and follows strict patterns, but Interaction Lab has made me realize that how a person writes code depends a lot on his or her own way of thinking.

The skills I mentioned above, learned and comprehended from Interaction Lab, play an important role in this world with emerging technology and innovation. Even though I didn’t choose this path, it’s always benefitting to have some knowledge about it.

This midterm project has also taught me that how close I am to perfection depends on how much effort I’m willing to put into what I do. We spent a lot of time on the 8th floor during the days before the presentation, staying there everyday until the lab closed. Even though it was tiring and time-consuming spending much time on the tiniest details, I realize that the more careful we are, the more likely our project won’t have problems during presentation. And sometimes if not paid attention to, details can have a huge impact on the overall effect. A good example would be the extra, single-color line of Greek letters that originally existed in my pattern.

Looking back, I think I cherish this process a lot. Not only did I put what I learned in the first half of the semester into practice, I also learned so much more about Processing in accordance to the project. I’m proud of myself for going through this. The midterm project has truly been a valuable experience.

//processing
import processing.serial.*;
Serial myPort; // Create object from Serial class
int val; // Data received from the serial port

int preval;

int counter, counter1, counter2;
float curvesin, curveSin;
float curveaddition, curveAddition;
boolean drawMagic=false;
int x, y;
PFont myFont;
// boolean makes drawmagic function possible to be controlled by key

int step = 15;
void setup() {

printArray(Serial.list()); // In this case, I can find the port number
String portName = Serial.list()[1];
myPort = new Serial(this, portName, 9600);

background(0);
smooth();// Draws all geometry with smooth (anti-aliased) edges.
size(600, 600);
counter = 0;
counter1 = 0;
counter2 = 40;
curvesin = 0;
curveSin=0;
curveaddition = 0.1;
curveAddition=0.1;//

// import font "AppleM yungjo" into the system
myFont = createFont("AppleM yungjo", 50);
textFont(myFont);
textAlign(CENTER, CENTER);
}

void draw() {

if ( myPort.available() > 0) { // If data is available,
val = myPort.read(); // read it and store it in val
println(val);

if (preval!=val) {
background(0);
}

if (val==1) {
pattern1();
}
if (val==2) {
pattern2();
}
}
preval=val;
}

void pattern1() {

pushStyle();
colorMode(RGB);
counter+=30; // increase 30 everytime,

noFill();
stroke(255, 100, 20);

//pushMatrix() and popMatrix() are used in conjuction with the other
//transformation functions and may be embedded to control the scope
//of the transformations.
// the γ cirlcle
pushMatrix();
fill(255, 255, 255, 20);
translate(width/2, height/2);
rotate(radians(counter+40));
text("γ", 30, 152); // x,y coordinates
rotate(radians(counter*0.75));
popMatrix();

// the stars consisted of trianangles
noFill();
stroke(255, 100, 20);
triangle(127, 200, 473, 200, 300, 500);
triangle(127, 400, 473, 400, 300, 100);
triangle(200, 132, 200, 472, 500, 308);
triangle(100, 300, 400, 473, 400, 124);

// call xy position to check the xy coordinates
//xyposition();
fill(0, 2);
// 0-rgb-color variable or hex value
//alpha-opacity of the fill
noStroke(); // Disables drawing the stroke (outline).
rect(0, 0, width, height);

// the servral circles
ellipse(width/2, height/2, 100, 100);
ellipse(width/2, height/2, 200, 200);
ellipse(width/2, height/2, 300, 300);
ellipse(width/2, height/2, 400, 400);

// the outside flower circle
// pushMatrix inside pushMatrix
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter+10));
stroke(255, 100, 20);
pushMatrix();//push another matrix
scale(1.92); // increase the scale by 1.92
translate(100, 20);//translate (100,20) to (0,0)
rotate(radians(110));//tranlate staring at (0,0) to 100 degrees
arc(0, 0, 28, 20, PI, PI+HALF_PI);
arc(28, -20, 28, 20, HALF_PI, HALF_PI+HALF_PI);
arc(28, 0, 28, 20, PI+HALF_PI, PI+HALF_PI+HALF_PI);
arc(0, -20, 28, 20, 0, HALF_PI);
popMatrix();
popMatrix();

// the inside smallerflower circle
// pushMatrix inside pushMatrix
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter2));
counter2+=140;
stroke(255, 100, 20);
pushMatrix();//push another matrix
scale(1.05);
translate(100, 20);//translate (100,20) to (0,0)
rotate(radians(110));//tranlate staring at (0,0) to 100 degrees
arc(0, 0, 28, 20, PI, PI+HALF_PI);
arc(28, -20, 28, 20, HALF_PI, HALF_PI+HALF_PI);
arc(28, 0, 28, 20, PI+HALF_PI, PI+HALF_PI+HALF_PI);
arc(0, -20, 28, 20, 0, HALF_PI);
popMatrix();
popMatrix();

// the inside larger flower circle
// pushMatrix inside pushMatrix
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter+10));
stroke(255, 100, 20);
pushMatrix();//push another matrix
scale(1.2);
translate(100, 20);//translate (100,20) to (0,0)
rotate(radians(110));//tranlate staring at (0,0) to 100 degrees
arc(0, 0, 28, 20, PI, PI+HALF_PI);
arc(28, -20, 28, 20, HALF_PI, HALF_PI+HALF_PI);
arc(28, 0, 28, 20, PI+HALF_PI, PI+HALF_PI+HALF_PI);
arc(0, -20, 28, 20, 0, HALF_PI);
popMatrix();
popMatrix();

// blue roatating circle inside
pushMatrix(); //
translate(width/2, height/2);
rotate(radians(frameCount));
stroke(0, 100, 255);
point(0, 70);
popMatrix();

//spinning rectangle

pushMatrix();
translate(width/2, height/2);

counter1 += 8;

rotate(radians(counter1));

//rotate in the counter randians, counter increases 30 everytime
// if counter exceeds 360, for example, 390, it euqals 30)
noFill();
stroke(255, 100, 20);
rect(0, 10, 20, 20);
popMatrix();

pushMatrix();
//pushMatrix() and popMatrix() are used in conjuction with the other
//transformation functions and may be embedded to control the scope
//of the transformations.
translate(width/2, height/2);// translate the triangle to the center of the picture
noStroke();
rotate(radians(counter*0.75));// make the triangle rotate
noFill();
stroke(255, 100, 20);
triangle(30, 10, 20, 30, 40, 30);
popMatrix();

// the flower part
pushMatrix(); //
translate(width/2, height/2);
rotate(radians(frameCount));
noFill();
stroke(0, 100, 255);
point(0, 75+(sin(curvesin)*20));
// point(x,y); float: x/y-coordinate of the point
//if it is just point(0,75); then it will just be a point rotate in a circle
//add sin(curvesin) in the y-coordinate,
//thus it will give people the illusion of the sin curve
curvesin += curveaddition;

popMatrix();

popStyle();
if (mousePressed) {
curveaddition = 0;
}
}

//xyposition is created to show the xy coordiantes of each points at the top left screen
// when don't need it, just comment out xypositon(), so that it will disaapear
//thanks for chirstian
void xyposition () {
x = mouseX;
y = mouseY;
fill(0);
//the place for rectangle
rect(0, 0, 100, 100);
fill(240);
text(x, 45, 20);
text(y, 45, 70);
}

void pattern2() {
//counter = 0;
textSize(40);//the Greek letters going around the circle
//different letters occupy different locations
colorMode(HSB);
fill(map(frameCount % 255, 0, 255, 90, 255), 255, map(frameCount % 255, 0, 255, 90, 255));
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter));
text("β", 0, 170);
text("γ", 30, 160);
text("δ", 60, 160);
text("η", 90, 140);
text("θ", 120, 130);
text("ξ", 140, 120);
popMatrix();

//pushMatrix();//pushMatrix() and popMatrix() are used in conjunction with the other
////transformation functions and may be embedded to control the scope of the transformations.
//translate(width/2, height/2);
//rotate(radians(counter));//rotate in the counter radians, counter increases 30 every time
////if counter exceeds 360, for example, 390, it equals 30
//text("β", 0, 200);
//popMatrix();

//pushMatrix();
//translate(width/2, height/2);
//rotate(radians(counter));
//text("γ", 30, 190);
//popMatrix();

//pushMatrix();
//translate(width/2, height/2);//translate the patterns to the center of the canvas
//rotate(radians(counter));
//text("δ", 60, 180);
//popMatrix();

//pushMatrix();
//translate(width/2, height/2);
//rotate(radians(counter));
//text("η", 90, 170);
//popMatrix();

//pushMatrix();
//translate(width/2, height/2);
//rotate(radians(counter));
//text("θ", 120, 160);
//popMatrix();

//pushMatrix();
//translate(width/2, height/2);
//rotate(radians(counter));
//text("ξ", 150, 150);
//popMatrix();

noFill();//the general shape of the patterns
ellipse(300, 300, 450, 450);
ellipse(300, 300, 425, 425);
ellipse(300, 300, 300, 300);
ellipse(300, 300, 121.132, 121.132);
ellipse(300, 300, 150, 150);
fill(0, 0, 0);
ellipse(300, 118.75, 62.5, 62.5);
ellipse(118.75, 300, 62.5, 62.5);
ellipse(300, 481.25, 62.5, 62.5);
ellipse(481.25, 300, 62.5, 62.5);
noFill();
ellipse(300, 118.75, 60, 60);
ellipse(118.75, 300, 60, 60);
ellipse(300, 481.25, 60, 60);
ellipse(481.25, 300, 60, 60);
noFill();
rect(193.935, 193.935, 212.132, 212.132);
quad(150, 300, 300, 150, 450, 300, 300, 450);
fill(255, 100, 20);

textFont(myFont);
textAlign(CENTER, CENTER);
textSize(70);

//stroke(map(frameCount % 255, 0, 255, 90, 255), 255, map(frameCount % 255, 0, 255, 90, 255));//the four letters occupying the four small circles
//textAlign(CENTER);
//textSize(70);

counter+=step; //the rotating, multiple rectangles in the middle
//println(counter);
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter));
noFill();
stroke(255, 100, 20);
rect(0, 10, 50, 50);

popMatrix();

counter+=step;//increase 15 every time
//println(counter);
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter));
noFill();
stroke(map(frameCount % 255, 0, 255, 90, 255), 255, map(frameCount % 255, 0, 255, 90, 255));
rect(5, 10, 50, 50);

popMatrix();

counter+=step;
//println(counter);
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter));
noFill();
stroke(map(frameCount % 255, 0, 255, 90, 255), 255, map(frameCount % 255, 0, 255, 90, 255));
rect(10, 10, 50, 50);

popMatrix();

counter+=step;
//println(counter);
pushMatrix();
translate(width/2, height/2);
rotate(radians(counter));
noFill();
stroke(map(frameCount % 255, 0, 255, 90, 255), 255, map(frameCount % 255, 0, 255, 90, 255));
rect(15, 10, 50, 50);
curveSin+=curveAddition;
popMatrix();
if (mousePressed) {
curveAddition=0;
}
}

//arduino
void setup() {
Serial.begin(9600);
//1 // number "1" //striing '1'//char value

// Serial.write()
// sends one byte as a number

// Serial.print()
// sends multiple bytes as a char
}

void loop() {
int val1 = analogRead(A0);
int val2 = analogRead(A1);

// just to test!
// Serial.print(val1);
// Serial.print(",");
// Serial.print(val2);
// Serial.println();

if (val1 < 40) {
//Serial.println(1);
Serial.write(1);
} else if (val2 < 120) {
Serial.write(2);
} else {
Serial.write(0);
}

}

One thought on “Interaction Lab Midterm Project: Magic Glove

  1. Read and Accepted. You have come a long way since the beginning of the semester when we were all struggling with lighting an LED. I see that you have learned a lot in this process, but I hope you will be more open to exploring ideas that are not necessarily practical in the real world. Sometimes it leads you nowhere, but at times it leads you to new possibilities that were never imagined. For example, your magical mandala looked so impressively similar to the original in the movies. But unlike the movie version, you have control over every line of code in your project. I hope you will feel more open to using different techniques to create this magical effect. Your writing of different technical aspects is becoming quite impressive and I encourage you to do more of it.

Leave a Reply