Ix Lab Emily Qiu Final Project: “A Mini Chemistry Lab” (Antonius)

Name of the Project: A Mini Chemistry Lab

Instructor: Professor Antonius

Partner: Grace

DESCRIPTION OF THE PROJECT

This project is both an educational and entertainment tool for kids who are interested in chemistry  experiments or actually whoever that is interested in chemistry experiments but fail to get access to the real lab can try this project. The project has two modes--experiment mode and magical mode in correspondence with the educational purpose and the entertainment use. For the experiment mode, basically, it is like a virtual experiment project but different from that, users can conduct partial physical interactions with the touchable laboratory equipments (currently what we have is a beaker) and view the process of the experiment step by step on the screen. In other words, users will be able to experience some interesting experiments in a both vivid and safe manner. As for the magical mode, it is more like a game as a treat to who has finished the experiment mode. Users will be able to experience a series of very weird but surprising “experiments” and you can check it out later in the demo. We have tried our best to make the project user-friendly because according to the user feedbacks for both our midterm projects, we think it is a very important point so we have designed many details to make the project become more “comfortable” for the user, such as changing cursor to indicate different functions or to add some little hints to let the user understand the rules more easily, which will be further explained in the following sections.

Grace is mainly responsible for the Arduino part and the physical part while I am mainly responsible for the processing part ❤️.

 

CONCEPTUAL DEVELOPMENT — IDEATION & RESEARCH

 

  • #Inspiration# My inspiration mainly comes from my own experience when I was little, actually not very little, like a Grade 4 or Grade 5. I was deeply attracted by chemicals and chemistry experiments during those days after our science teacher showed us how to make oxygen via heating KMnO4. In addition, my grandma used to be a high school chemistry teacher so she really encouraged me to explore the amazing world of chemistry. However, when I asked them whether could I operate some experiments by myself, both of them said no because they thought it was too dangerous to let a kid to touch those chemicals. I was super depressed and my curiosity even grew bigger when they rejected my request. So my grandma finally agreed to let me conduct some experiments that are very safe and can be conducted with household materials such as sugar or vinegar. Still, she would never let me to do some “potentially harmful” things such as using fire, the most frequent thing she asked me to do was to measure a certain volume of liquid. I was confused at the beginning about why it was important to read the accurate volume because if I was allowed to do one on my own, I would just mix those liquids together regardless of their proportion as it was easy and convenient. Then one day, my grandma told me that you might see reading volume as  a boring task but in fact, it is really a crucial ability that I should be equipped with because different proportions of chemicals can result in completely different reactions and some of them can be rather dangerous such as explosion. Thus, the ability to read accurate volume and get to understand the importance of the concept “proportion” to chemical experiments is something we want to emphasize, so this is also the focus of our project. We want our children to feel the charm of chemical experiments in both a safe and entertaining manner and we hope that they can learn something like what I did when I was a kid.

 

  • #References#  
  1. YouTube Virtual Chemistry Lab Video: Our project is somewhat similar to a VR lab because users cannot really see the actual, physical reactions but instead, they view the experiments in a relatively virtual way. So we get some inspiration from this video about how to build interactions between physical movements and the changes appearing in the screen. https://www.youtube.com/watch?v=M7kaLX4_qL0
  2. Processing organization: when I was thinking about the interaction design for Processing part, I refer to this website for help:https://processing.org
  3. Pinterest: my images and some visual elements are credited to this website: https://www.pinterest.com
  4. Some coding example reference:https://forum.processing.org/two/discussion/21785/bubbles-blinking(bubbles) https://www.dfrobot.com/wiki/index.php/SHARP_GP2Y0A41SK0F_IR_ranger_sensor_(4-30cm)_SKU:SEN0143 (Sensor)

 

  • #Target Audience# As what I have mentioned before, my target audience are mainly kids who are interested in chemistry experiments but are too young to operate those real ones on their own. Actually it is more like a puzzle game especially its magical mode, so I would say that the range of audience is actually even broader.

 

TECHNICAL DEVELOPMENT–DESIGN & IMPLEMENTATION

 

  • #Flow Chart# 

Before implementation, we have drawn many drafts and have discussed about many possible plans, this is our final version of flow chart showing basically how our project works.

 

  • #Materials# 
  1. Distance Sensor
  2. Breadboard
  3. All kinds of wires in the Knit
  4. Arduino
  5. Tape and two measuring pots

 

  • #Arduino and Physical Part# 

The coding for the Arduino part is not very complex which is basically used to read the sensor and transmit the data to the processing.

The most difficult part might be how to build proper connections between the data provided by distance sensor and the volume of the liquid. The sensor we use is a distance sensor, though at the very beginning we planned to use water pressure sensor or weight sensor,

it turned out that either ER doesn’t have one or it is just too complex so that our target users might not be able to do that. So in the end, we decide to use distance to test the volume of the liquid since our measuring cup is a standard cup. So here is a sketch showing how our sensor works. As is shown in the code, there is a conversion which though seems to be simple, in fact costs a lot of time and trials to figure out how to make those two data consistent.

 

  • #Processing# 

To begin with, we have approximately 600 lines of codes for processing because we have six pages in total with animation and many images. I feel really proud when I scan them after finishing the project 😊!

Home page and the intro page:

For the home page, we have utilized the function “cursor”, so when the user moves the mouse to the title, the title will changes into color orange and when the user moves the mouse to the “START”, the color of the font will become orange as well and the cursor will be changed from arrow to hand so that users will know that it can be clicked.

As for the button “NEXT” in the intro page, besides the cursor function, it is actually a little animation. The little “NEXT” will gradually become bigger and when it is big enough, it will becomes small again. So it is a simple animation that is aimed to attract people’s attention so that users will easily notice that button and click it to continue.

 

Experiment Mode:

The experiment has two steps and the formulas we chose are as follows:

We have spent a lot of time on selecting a proper formula because we want a reaction that has both an apparent effect such as color change and a strict requirement for the volume of the chemicals involved. In the end, we chose the reaction between H2O2 and FeCl2 because depending on the portion of the H2O2, the reaction is actually different. For the first step, when H2O2:FeCl2=1:2, the result is only a color change. Fe3+is produced so the color of the liquid changes from light green to dark orange-red. While for the step2, when H2O2 is more than half of the FeCl2, there will emerge oxygen and users will see bubble floating out.

This is how the step 1 looks like:

According to the instructions, the user will first click on the button FeCl2 and measure 200ml water  and add it to our measuring cup. It won’t work when the volume is not correct. To make if easier for users to understand, when the “chemical” is FeCl2, there will be a little red circle beside the image like the screenshot on the left to inform the audience which “chemical” they are adding to the cup. Then the user will follow the instructions to add H2O2 and click the cup to mix them together. The color of the liquid will change when the reaction is finished like what is shown in the video below. (There is also demonstration in the demo)

 

In order to control which “chemical” is selected, we have set up two variables c1 and c2. After clicking each chemical, the value of c1 and c2 will change accordingly so that we can control which chemical is adding to the cup and demonstrate the changing volume of the liquid.

This is the part in the mouseClicked() function attached in the end of the code

 

As for the step2, the page is generally similar to the step 1, the biggest difference is the bubble part. When the user clicked the cup to mix the two chemicals together, there will be bubbles floating out like this:

For the bubbles, grace and I have struggled a lot about the classes. We have referred to many reference and we finally figured out how to do it (reference is in the reference section above).

The use of boolean is also one of our progress which we found is sometimes easier than if condition.

 

Magical Mode:

When the user entered the magical mode, it is more like a tricky little game.

User can click on different “chemicals” such as carrot juice, then there will be a magical “reaction” in the cup like what the image shows–carrot juice+milk=a bunny. (for fish it is a kitty while for banana it is a tiny monkey).

For the “?”, when the user clicks it, it is more like a prank 😂. The user will see his/her photo in the cup like this:

For this part, I mainly utilized the technique we learnt in class to use camera. However, it is a little pity that the size of the capture camera is fixed, so I cannot adjust it to the exact size of the cup. But I think it is still very interesting and surprising for users according to the feedback I have received.

 

DEMO

The video is a little bit too big so I have compressed it maybe a little bit too much. Here is a link to the complete one in goole drive:

https://drive.google.com/open?id=1UEAl4e5WDfIArRzVjTQORcskCphrRDkC

 

USER TESTING & LESSONS LEARNED

Though Grace and I have classes during the IMA show so we didn’t make it to the show which is very a pity, but we have conducted user testing among our friends both during our fabrication and after we finished the whole project.

Most our users find it interesting to see the reaction appearing on the screen, according to Luisa, one of the Freshman, she said that it was great fun to explore chemical experiments like this and she hoped that we could have more reactions in the future, which is also our expectation if we have more time. Also, many of them loved our magical mode especially the camera part because it was surprising for them and they found it funny to become “a result of an reaction”. Better than my midterm, this time all my users found the instructions easy to follow and they can finish each step without my reminding, which I think is a good sign of my progress in terms of making “user-friendly products”. They also loved our interface which is according to Doris, “cute and cute and cute”.

One of the suggestions we received is that maybe there should be more reactions for users to choose from and the formula might be a little to complex for kids. We will definitely work on that but it is a little hard to find a formula that both has an apparent effect and also an apparent visual effect depending on different volume, but we will make more efforts on it. Another suggestion is that maybe there can be more interactions and Grace and I are considering about adding “mixing by hands” instead of clicking the cup.

The lessons we learnt are mainly:

  1. User-friendly design is very important. Always think for your user, especially your target audience.
  2. Clear instructions and interface design that appeals to your users.
  3. Interesting interactions and give your user a little surprise.
  4. Organize the coding! Especially when you have a lot of different individual pages!
  5. When one way doesn’t work (the water pressure sensor), don’t panic, try another! 😁
FINAL REMARKS

Thanks for everything for this amazing semester and I really enjoy it! I like the way how I build interactions between physical touchable things and my computer! It’s like breaking down the wall between virtual world and the reality and luckily, I am the one who “build bridges” between these two worlds.

I will continue my journey in IMA and in the end  thank you, Professor Antonius for being the best professor! Patient and kind! ❤️ Hope to see you in the future! And also, thanks for all the amazing fellows and classmates!❤️

Arduino Part----------------------------------------------------------------------------------

//connect gp2d120x to A1
#define pin A1

void setup () {
        Serial.begin (9600);
        pinMode(pin, INPUT);
   
}

void loop () {
        uint16_t value = analogRead (pin);
        double distance = get_IR (value); //Convert the analog voltage to the distance
        int v=10-distance;
        Serial.write (v);

        delay (500);                            //Delay 0.5s
        
}

//return distance (cm)
double get_IR (uint16_t value) {
        if (value < 16)  value = 16;
        return 2076.0 / (value - 11.0);
       
        
}

Processing Part----------------------------------------------------------------------------------

import processing.serial.*;
import processing.video.*; 
Capture cam;

//String myString = null;
Serial myPort;
//int NUM_OF_VALUES = 1;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ // This is the number of values you are receiving. 
//int[] sensorValues;      /** this array stores values from Arduino **/

ArrayList<Bubble> bubbles;
int add = 0;
int initial = 10;


PImage bg1,bg2,bg3,cup1,cup2,formula1,formula2,formula3,cat,cont1,cont2,cont3,ins,carrot,fish,banana,mystery,kitty,monkey,rabbit;
PFont myFont;

int mode=0;
float u=25;
float v1=0,v2=0;
float speed=0;
int c1,c2,c3,c4=0;
int c,b,f,m=0;


void setup(){
 fill(255);
 size(1280,780);
 bubbles = new ArrayList<Bubble>();
 cam = new Capture(this, 320, 240);
 cam.start(); 
  
 bg1=loadImage("bg.jpg");
 bg1.resize(1280, 780);
  
 bg2 =loadImage("bg2.jpg");
 bg2.resize(1280, 780);
 
 bg3 =loadImage("bg3.jpeg");
 bg3.resize(1280, 780);
 
 cup1=loadImage("cup.png");
 cup2=loadImage("cup2.png");
 
 formula1=loadImage("formula.png");
 formula2=loadImage("formula2.png");
 formula3=loadImage("formula3.png");
 
 cat=loadImage("cat.jpg");
 cat.resize(600,780);
 cont1=loadImage("cont1.png");
 cont2=loadImage("cont2.png");
 cont3=loadImage("cont3.png");
 
 ins=loadImage("instruction.png");
 ins.resize(600,500);
 
 //magic mode images
 
 carrot=loadImage("carrot.jpg");
 carrot.resize(110,110);
 
 banana=loadImage("banana.jpg");
 banana.resize(110,110);
 
 fish=loadImage("fish.jpg");
 fish.resize(110,110);
 
 mystery=loadImage("mystery.jpg");
 mystery.resize(110,110);
 
 kitty=loadImage("kitty.jpg");
 kitty.resize(270,340);
 
 monkey=loadImage("monkey.jpg");
 monkey.resize(270,340);
 
 rabbit=loadImage("rabbit.jpg");
 rabbit.resize(270,340);
 

 
 
 myPort= new Serial(this, Serial.list()[3], 9600);
  //setupSerial();
 
}

void draw(){

//updateSerial();
//printArray(sensorValues);


PFont font;
font = loadFont("Chalkduster-48.vlw");



//page 1 >>cover--------------------------------------------------------------------------
if(mode==0){

  
  image(bg1,0,0);
  fill(255);
  textFont(font, 40);
  text("START",1000,250);
  textFont(font, 70);
  text("#A MINI CHEMISTRY LAB#", 80, 150);

  
if(990<mouseX&& mouseX<1150 && 220<mouseY && mouseY<260){
  fill(#E88A2C);
  textFont(font, 40);
  text("START",1000,250);
  cursor(HAND);
}else if(70<mouseX && mouseX <1153 && 85<mouseY && mouseY<165){
  fill(#E88A2C);
  textFont(font, 70);
  text("#A MINI CHEMISTRY LAB#", 80, 150);

}else{
  fill(255);
  textFont(font, 40);
  text("START",1000,250);
  cursor(ARROW);
  fill(255);
  textFont(font, 70);
  text("#A MINI CHEMISTRY LAB#", 80, 150);
  textFont(font, 30);
}
}

//page 2 >>intro page-----------------------------------------------------------------------
if(mode==1){
  
  image(bg2,0,0);
  cursor(ARROW);
  
  fill(#F293C1);
  textFont(font, 70);
  text("WHAT CAN I DO : ) ?", 80, 150);
  
  textFont(font, 25);
  fill(0);
  text("* Welcome to this amazing tiny lab!!!",80,220);
  text("* Here you can explore some interesting chemical experiments safely, and",80,270);
  text("also acquire some basic knowledge about how to operate an experiment properly",80,320);
  text("* By following the instructions step by step, you will first complete some",80,370);
  text("real experiments and if you pass the exercise, you will enter a magical mode!",80,420);
  fill(#F293C1);
  text("* Interacting with physical laboratory equipments and check the process of ",80,470);
  text("your experiment on the screen! Try this project right now!",80,520);
  fill(0);
  textFont(font,u);
  text("<NEXT>",1000,620);
  if(u<=50){
    u=u+0.3;
  }else if(u>50){
    u=25;
  }
  if(900<mouseX&& mouseX<1200 && 590<mouseY && mouseY<640){
  fill(255);
  textFont(font,u);
  text("<NEXT>",1000,620);
  cursor(HAND);
}
}


//page 3 >>experiment 1---------------------------------------------------------------------
if(mode==2){
  
  if (100<mouseX&& mouseX<188 && 200<mouseY && mouseY<286 ||100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487){
  cursor(HAND);
    
  }else{
    cursor(ARROW);
  }
  
  background(bg2);
  image(cup1,100,200);
  image(cup2,100,400);
  image(formula1,250,50); 
  fill(0);
  textFont(font,23);
  text(">>EXPERIMENT 1",30,95);
  text("FeCl2",100,310);
  text("H2O2",100,510);
  textFont(font,18);
  text("(containing H+)",100,330);
  fill(#7CE3AC);
  text("blue-green liquid",100,360);
  fill(255);
  text("transparent liquid",100,540);
 
   
  //instruction bubble
  image(ins,270,150);
  fill(0);
  textFont(font,16);
  text("1. Click FeCl2, add 200 ml liquid to the cup,",370,330);
  text("click the image again when stop filling",370,360);
  text("2. Click H2O2, add 100 ml liquidto the cup",370,390);
  text("3. Click the cup to mix them together",370,420);
  text("4. Observe the color and click NEXT",370,450);
  
  noFill();
  stroke(0);
  rectMode(CORNER);
  rect(900, 230, 280, 320,0,0,60,60);
  fill(#7CE3AC);
  
  rectMode(CORNERS);
  noStroke();
  rect( 900, 550-v1,1180,550,0,0,60,60);
  strokeWeight(1);
  
  rectMode(CORNER);
  noFill();
  stroke(1);
  rect(900,230,280,v2);
  
  if(c1==1){
  while(myPort.available()>0){
    v1=myPort.read()*20;

  }

  }else
  
  if(c2==1){
   while(myPort.available()>0){
    v2= 320-myPort.read()*30;
    }
    if(v2>500){
      text(0,0,255);
      textSize(32);
      print("WARNING", 900,550);
    }
  }else
  
  if(v1>50 && v2>50 && c3==1){
    rectMode(CORNERS);
    noStroke();
    fill(#F05933);
    rect( 900, 550-v2,1180,550,0,0,40,40);
  }
  
  //next button
  
  fill(0);
  textFont(font,40);
  text("<NEXT>",1000,700);
  if(950<mouseX&& mouseX<1100 && 680<mouseY && mouseY<720){
  fill(255);
  textFont(font,40);
  text("<NEXT>",1000,700);
  cursor(HAND);
}

  
}

//page 4 >>experiment 2------------------------------------------------------------------

if(mode==3){
  

  background(bg2);
  image(cup1,100,200);
  image(cup2,100,400);
  image(formula2,250,50); 
  fill(0);
  textFont(font,23);
  text(">>EXPERIMENT 2",30,95);
  text("FeCl2",100,310);
  text("H2O2",100,510);
  textFont(font,18);
  text("(containing H+)",100,330);
  fill(#7CE3AC);
  text("blue-green liquid",100,360);
  fill(255);
  text("transparent liquid",100,540);
  
  //next button
  fill(0);
  textFont(font,40);
  text("<NEXT>",1000,620);
  if(900<mouseX&& mouseX<1200 && 590<mouseY && mouseY<640){
  fill(255);
  textFont(font,40);
  text("<NEXT>",1000,620);
  cursor(HAND);
}

   //instruction bubble
  image(ins,270,150);
  fill(0);
  textFont(font,16);
  text("1. Click H2O2, add another 100 ml liquid,",370,330);
  text("click the image again when stop filling",370,360);
  text("2. Click the cup to mix them together",370,390);
  text("3. Observe what happens to the liquid",370,420);
  text("4. Click NEXT",370,450);
  
  rectMode(CORNERS);
    noStroke();
    fill(#F05933);
    rect( 900, 550-v2,1180,550,0,0,60,60);
    if(c4==1){
  bubbles.add(new Bubble(random_generator()));
 
  //If bubble gets to the top it is removed
  for (int i = 0; i < bubbles.size(); i++){
    Bubble b = bubbles.get(i);
    b.ascend();
    b.display();
    if (b.top()){
      bubbles.remove(i);
    }
 
  }

}

  rectMode(CORNER);
  noFill();
  stroke(1);
  rect(900,230,280,v1);
  image(ins,270,150);
  noFill();
  stroke(0);
  strokeWeight(1);
  rectMode(CORNER);
  rect(900, 230, 280, 320,0,0,60,60);
  if (100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487){
  cursor(HAND);
    
  }else{
    cursor(ARROW);
  }
  if (c2==1){
    while(myPort.available()>0){
    v1= 320-myPort.read()*30;

  }
  
  
  
  
}


//page 5 >> magical mode intro------------------------------------------------------------------


if(mode==4){
  
  cursor(ARROW);
  background(bg3);
  image(cat,0,0);
  image(cont1,720,80);
  image(cont2,900,150);
  image(cont3,640,190);
  
  fill(0);
  textFont(font,18);
  text("Here you can experience some fantastic(weird) experiments!",640,400);
  text("Trust me they are amazing! Follow the instructions",640,450);
  text("and let's go ~~~~~",640,500);
  
   //next button
  
  fill(0);
  textFont(font,u);
  text("<NEXT>",1000,700);
  if(u<=50){
    u=u+0.3;
  }else if(u>50){
    u=25;
  }
  if(950<mouseX&& mouseX<1100 && 680<mouseY && mouseY<720){
  fill(255);
  textFont(font,u);
  text("<NEXT>",1000,700);
  cursor(HAND);
}
 

}



//page 6 >> magical mode experiment 1------------------------------------------------------------------

if(mode==5){
  
  if (100<mouseX&& mouseX<200){
  if(150<mouseY && mouseY<250||310<mouseY && mouseY<400){
    cursor(HAND);
  }else if(450<mouseY && mouseY<550||600<mouseY && mouseY<700){
    cursor(HAND);
  }
    
  }else{
    cursor(ARROW);
  }
  
  background(bg2);
  image(formula3,250,50); 
  
  println(mouseX,mouseY);
  image(carrot,100,150);
  image(fish,100,300);
  image(banana,100,450); 
  image(mystery,100,600);
  fill(0);
  textFont(font,23);
  text(">>MAGIC MODE ",30,95);
  
  textFont(font,18);
  text("Carrot Juice",100,280);
  text("Fish",100,430);
  text("Banana",100,580);
  text("Mystery",100,730);

  //instructions 
  image(ins,270,150);
  fill(0);
  textFont(font,16);
  text("Here is a cup of 400ml milk",370,330);
  text("Choose an element from the left",370,360);
  text("Click the cup to mix them together",370,390);
  text("Check what happens!!!",370,420);
  text("You can try other choices as well~~~",370,450);
  
  
  noFill();
  stroke(0);
  rectMode(CORNER);
  rect(900, 230, 280, 320,0,0,60,60);
  fill(#7CE3AC);  
 
  rectMode(CORNER);
  noStroke();
  fill(#FCFAFA);
  rect(900, 320, 280, 230,0,0,60,60);


  if(c==1){
  image(rabbit,907,200);
  }if(b==1){
  image(monkey,907,200);
  }if(f==1){
  image(kitty,907,200);
  }if(m==1){
     if (cam.available()) { 
   cam.read(); 
  } 
  pushMatrix();
  translate(cam.width, 0);
  scale(-1,1);
  tint(250,144,63);
  image(cam, -880, 300);
  popMatrix();
  noTint();
  
  
  
  
  }
  
 

  
  //next button
  
  fill(0);
  textFont(font,40);
  text("<DONE>",1000,620);
  if(900<mouseX&& mouseX<1200 && 590<mouseY && mouseY<640){
  fill(255);
  textFont(font,40);
  text("<DONE>",1000,620);
  cursor(HAND);
}

  
}
}

}

int random_generator(){
  int r = int(random(10, 100));
  return r;
  }
//MouseClicked functions------------------------------------------------------------------
void mouseClicked(){
  if (mode==0){
    if(990<mouseX&& mouseX<1150 && 220<mouseY && mouseY<260){
      mode=1;
    }
  }
      if (mode==1){
    if(900<mouseX&& mouseX<1200 && 590<mouseY && mouseY<640){
      mode=2;
    }
  }
  
  //page3
if(mode==2){
  if (100<mouseX&& mouseX<188 && 200<mouseY && mouseY<286 && c1==0 ){
    c1=1;
  
    
  }else if(100<mouseX&& mouseX<188 && 200<mouseY && mouseY<286 && c1==1){
    c1=0;
    }
    if (100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487 && c2==0 ){
    c2=1;
  
    
  }else if(100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487 && c2==1){
    c2=0;
    }
    else if(950<mouseX&& mouseX<1100 && 680<mouseY && mouseY<720){
      mode=3;
}
if(c3==0 && 900<mouseX&& mouseX<1180 && 230<mouseY && mouseY<550 ){
   c3=1;
 }
}

if(mode==3){
  if(900<mouseX&& mouseX<1200 && 590<mouseY && mouseY<640){
      mode=4;

}
if (100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487 && c2==0 ){
    c2=1;
  
    
  }else if(100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487 && c2==1){
    c2=0;
    }
if(c4==0 && 900<mouseX&& mouseX<1180 && 230<mouseY && mouseY<550 ){
   c4=1;
  }else if(c4==1 && 900<mouseX&& mouseX<1180 && 230<mouseY && mouseY<550){
    }
}

if(mode==4){
  if(950<mouseX&& mouseX<1100 && 680<mouseY && mouseY<720){
    mode=5;
  }
    if (100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487 && c2==0 ){
    c2=1;
  }else if(100<mouseX&& mouseX<188 && 400<mouseY && mouseY<487 && c2==1){
    c2=0;
    }
 
}



if(mode==5){
  if (100<mouseX&& mouseX<200 && 150<mouseY && mouseY<250){
    c=1;
    b=0;
    f=0;
    m=0;
  
  }else if (100<mouseX&& mouseX<200 && 310<mouseY && mouseY<400){
    c=0;
    b=0;
    f=1;
    m=0;
  }else if(100<mouseX&& mouseX<200 && 450<mouseY && mouseY<550){
    c=0;
    b=1;
    f=0;
    m=0;
    }else if(100<mouseX&& mouseX<200 && 600<mouseY && mouseY<700){
    c=0;
    b=0;
    f=0;
    m=1;
}  else if(900<mouseX&& mouseX<1200 && 590<mouseY && mouseY<640){
    mode=0;
  }
}

}

//bubbles
class Bubble{ 
  float x;
  float y;
  float diameter;
  float yspeed; 

  Bubble(float tempD){
    x = random(920,1160);
    y = 230;
    diameter = tempD;
    yspeed = random(2.5, 10.5);
  }
 
  //moves bubble upwards, and randomly from side-to-side
  void ascend(){
    y = y - yspeed;
    x = x + random(-2, 2);
  }
 
  //displays bubble
  void display(){
    noStroke();
    fill(231, 254, 255, 200);
    ellipse(x, y, diameter, diameter);
  }
 
  //checks if bubble is at the top of the screen
  boolean top(){
    if (y < 0){
      return true;
    }
    else{
      return false;
    }
  }
}

Ix Lab Recitation 10: Media Controller (Antonius)

Name of the Recitation: Media Controller “Photos Of Cecilia & Emily”

Professor: Professor Antonius

Partner: Cecilia Cai

 

GOALS FOR THIS RECITATION:

-Work in pairs in order to create a Processing sketch that controls media elements (images, videos and audio) using a physical controller made with Arduino..

-Document the work to my blog.

-Write a reflection on this theme.

-❤️

 

The Process of making a media controller

Cecilia and I decided to use potentiometers as physical values to create a media controller named “Photos of Cecilia & Emily” because we two are very good friends and we both love taking photos. We have literally tons of photos so we thought: “Why don’t we make something to display them in our own unique way?” So our idea was basically “a display window” for some of our favorite photos and it is combined with some audios.

 

  • How The Project Works:

There are two potentiometers on the breadboard and they can respectively manipulate the xPos and yPos of a happy little circle on processing. The color of that little circle keeps changing because we set the code “fill(random(255), random(255), random(255))” and it will moves according to the value sent from those two potentiometers. Though the boundaries are invisible, the canvas is actually divided into four different areas. When the circle moves into a certain area, for example, the right-up corner which is marker with “area1”, the background image will immediately be changed to “photo1” and “music1” will be played. Similarly, when the circle moves to different areas, the corresponding image will be displayed and the corresponding music will be played.

(music source: Apple iTunes)

 

  • How The Project Looks Like In The End:

The physical part looks like this (ignore the button because we first planned to use it but in the end we gave up):

 

Here is a video clip showing how it works:

 

  • Problems & Lessons

For the serial communication between Arduino and Processing, we mainly referred to the knowledge we learnt in last week’s workshop.

I am mainly responsible for the circuit (Arduino) part and the image effect in processing while Cecilia is mainly responsible for the sound effect.

For the image effect, the method we used was basically by using a big if-condition to manipulate different effects. The location of our little circle was “(xPos,Ypos)” so the statement of our if-condition was meant to divide the canvas into four parts. However, as I tried to put all the instructions directly under the if-condition, it turned out to not work at all. I didn’t really understand the reason behind but I decided to try another method. Instead of putting all the instructions like the change of image as well as sound, I can add another if condition below by setting a new variable. I then set a variable “z” whose value would be changed when different statement was met, and when it held different values, different effects would be generated under the control of a new if-condtion like this:

(ignore the audio’s name because we made them just for fun they are actually amazing music works downloaded from iTunes.

In this way, we separated the condition statements with the controlling effects and the image effect could finally work properly after we reorganized our codes like this. I don’t know whether this method really made a difference or there was just something wrong with my first version of codes, but this method can really makes my codes more organized and clear.

The biggest problem occurred in the sound effect controlling. There were several stages for our problem. The first was the background music. At the very beginning, we decided to use a background music. Cecilia’s idea was “to have a background music playing all the time [when the button is pressed], and have the point generate four different audio pieces as it moves”. However, when we tested this first attempt, we kept getting error reports of “Index out of array”. So we turned to one of the assisting fellows for help, who told us that the issue was each time the point was moved to a specific block, an audio piece would start to play, but as we didn’t set any instructions to let the previous music stop, all the sound files were actually playing together, which leads to a terrible catastrophe for the program– overloading. Moreover, the background music made things even worse as our original design was to play the background music when the button is pressed, and to pause when the button was pushed again. This was also something accounting for our problem as the background music would keep pausing and playing if the button is pressed, so the music would play intermittently, making the sound effect even more confusing. In order to solve this problem, we finally removed the background music to simplify the sound effect and added a “stop()” function to the sketch.

However, problems still existed  and the error message jumped out again when the circle moved to different blocks. We thought that the reason might be because the “stop()” instruction didn’t function effectively, as sometimes it was called when the music piece was not playing which might caused trouble for the operation of the code. We had tried several approaches to solving this problem and we even tried to download another library but all failed. In the end, we went to the lab and Professor Rudi came to our aid. What he suggested us to do was to set some Boolean variables, each variable was linked to one audio file and determined whether it was playing or not. When the audio is playing, the corresponding variable would appeared to be true, and when it stopped, it was false. So for each block, it basically looked like this:

Though it might seem to be a little complicated because it was super long and we had to repeat it for four times. However, it worked and this was my first time to use boolean, which was a new breakthrough for me and I really enjoyed this experience.

 

Reflection

For our project, the way how user can use the media controller to interact with computational art is relatively simple. The user simply switches those potentiometers to produce media arts. We basically use the values sent from potentiometers as an input data to control the media. When it comes to approaches to using physical computation to create interactive media art, I think they are indeed very diverse and there are just too many ways of doing it. Probably one of the simplest way is to use a simple move to create some kind of data and send those values to the computer in order to generate some art depending on those variables. For example, by switching the potentiometer or pushing some buttons. There are also some similar ones but seem to be more “advanced”–to detect and analyze human’s different moves, such as a gesture or even a simile, which means that there is no need for physical touches but instead, the user’s hands are “free” from the physical interface and merely had to “let the computer to see his/her moves”. I think this kind of interaction is even more natural and magical as the user seem to “have no connections” with the installation but could create an art with his/her interaction. I can’t imagine how cool it is if we can let a user to create a piece of art by only smiling–a piece of drawing whose lines and colors are controlled by the different ways of smiling, a song that is made of musical notes corresponding to the different expressions of his/her smiles and even a video of his/her smile with some interesting video editing. In fact, besides smiles and gestures, these visual input, sounds and other senses can all produce media art and they are often combined together. Just like the example of Messa di Voce, created by this article’s author in collaboration with Zachary Lieberma in this week’s reading, Computer Vision for Artist and Designers. This project  “uses whole-body vision-based interactions similar to Krueger’s, but combines them with speech analysis and situates them within a kind of projection-based augmented reality” which “uses a set of vision algorithms to track the locations of the performers’ heads [and] also analyzes the audio signals coming from the performers’ microphones”. In this way, users can use different senses to interact with one device to create his/her own art, which is very amazing and makes users feel like they have limitless possibilities to create art with the help of digital computer. They are making the art, they are interacting with the art, they are enjoying the art and they are also a critical part of the media art.

 

//Media Controller <<Arduino>>----------------------------------------------

void setup() {
  Serial.begin(9600);
  pinMode(2, INPUT);
}

void loop() {
  int sensor1 = analogRead(A0)/4;
  int sensor2 = analogRead(A1)/4;
  int button = digitalRead(2);

  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(",");
  Serial.print(button);
  Serial.println(); // add linefeed after sending the last sensor value

  // too fast communication might cause some latency in Processing
  // this delay resolves the issue.
  delay(100);
}







//Media Controller <<Processing>>----------------------------------------------



import processing.sound.*;
import processing.serial.*;

SoundFile bgm;
SoundFile zombie;
SoundFile sefs;
SoundFile park;
SoundFile space;

String myString = null;
Serial myPort;

int NUM_OF_VALUES = 3;   
// This is the number of values you are receiving. 
int[] sensorValues;      /** this array stores values from Arduino **/

int xPos = width/2; 
int yPos = height/2;
int z;

PImage img;
//PFont myFont;
PFont font;
boolean zombiePlaying, sefsPlaying, parkPlaying, spacePlaying;

void setup() {
  size(600, 600);
  img=loadImage("pink.jpg");
  img.resize(600, 600);

  setupSerial();
  z = 0;

  bgm = new SoundFile(this, "bgm.mp3");
  zombie = new SoundFile(this, "1.mp3");
  sefs = new SoundFile(this, "2.mp3");
  park = new SoundFile(this, "3.mp3");
  space = new SoundFile(this, "4.mp3");

  font = loadFont("Chalkduster-48.vlw");
}


void draw() {
  updateSerial();
  printArray(sensorValues);

  xPos = int(map(sensorValues[0], 0, 255, 0, width));
  yPos = int(map(sensorValues[1], 0, 255, 0, height));

  if (xPos > 0 && xPos < width/2 && yPos > 0 && yPos < height/2) {
    z=1;
  } else if (xPos > width/2 && xPos < width && yPos > 0 && yPos < height/2) {
    z=2;
  } else if (xPos > 0 && xPos < width/2 && yPos > height/2 && yPos < height) {
    z=3;
  } else if (xPos > width/2 && xPos < width && yPos > height/2 && yPos < height) {
    z=4;
  }


  if (z==1) {
    img=loadImage("1.jpg");
    img.resize(600, 600);
    if (!zombiePlaying) {   
      zombie.play();
      zombiePlaying = true;
    }
    if (sefsPlaying) {  
      sefs.stop();
      sefsPlaying = false;
    }
    if (parkPlaying) {  
      park.stop();
      parkPlaying = false;
    }
    if (spacePlaying) {  
      space.stop();
      spacePlaying = false;
    }
  } else if (z==2) {
    img=loadImage("2.jpg");
    img.resize(600, 600);
    if (!sefsPlaying) {  
      sefs.play();
      sefsPlaying = true;
    }
    if (zombiePlaying) { 
      zombie.stop();
      zombiePlaying = false;
    }
    if (parkPlaying) {  
      park.stop();
      parkPlaying = false;
    }
    if (spacePlaying) {
      space.stop();
      spacePlaying = false;
    }
  } else if (z==3) {
    img=loadImage("3.jpg");
    img.resize(600, 600);
    if (!parkPlaying) {
      park.play();
      parkPlaying = true;
    }
    if (zombiePlaying) {
      zombie.stop();
      zombiePlaying = false;
    }
    if (sefsPlaying) {
      sefs.stop();
      sefsPlaying = false;
    }
    if (spacePlaying) 
      space.stop();
    spacePlaying = false;
  }
}  
if (z==4) {
  img=loadImage("4.jpg");
  img.resize(600, 600);
  if (!spacePlaying) {
    space.play();
    spacePlaying = true;
  }
  if (zombiePlaying) {
    zombie.stop();
    zombiePlaying = false;
  }
  if (sefsPlaying) {
    sefs.stop();
    sefsPlaying = false;
  }
  if (parkPlaying) {
    park.stop();
    parkPlaying = false;
  }
}

image(img, 0, 0);


fill(255);
textFont(font, 55);
text("#PHOTO#", 100, 300);

strokeWeight(32);
stroke(random(0, 255), random(0, 255), random(0, 255));
point(xPos, yPos);
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 3 ], 9600);

  myPort.clear();

  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = 'n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Ix Lab Recitation 9: Serial Communication Workshop (Antonius)

Name of the recitation: Serial Communication Workshop

Professor: Professor Antonius

Workshop Fellow: Cici & Nick

 

GOALS FOR THIS RECITATION:

-Attend a workshop.

-Document the work I did during the workshop.

-Explain how will it help me for my final project.

-❤️

 

The workshop–Serial communication

I was most interested in serial communication because I still felt a little confused about transmitting multiple data by using serial communication. For my final project, I may want to use multiple sensors to manipulate my processing interface so I want to strengthen my ability of using serial communication through this workshop.

We first had a review of what we have learnt about serial communication before, including some key concepts such as bit, byte, UART, USB, Data Rate, Port and some data types such as int=integral, char=character, string=a bunch of characters. An important point Nick stressed a lot is the difference between Serial.write() and Serial.print()– Serial.write() only writes bytes while Serial.print() prints ASCLL or UnicodeI always messed up with this difference when I wrote codes but now I feel that I am clearer about this point (it seems that Serial.print() is a better choice🙈 as it can print strings including 0101)

 

Then we came to the first example–Arduino To Processing.

The exercise is to use the data transmitted from two potentiometers and a button to control a little happy circle in processing. By switching two potentiometers, the circle will move upward/downward or left/right and by pushing the button, the circle will present a random color. This was my first time to let processing receive multiple data to control the object in processing. I felt very excited when I managed to make it work.

Here is how the circuit looks like:

 

How the circuit works:

 

Then we came to the second example–Processing To Arduino.

For this circuit, it is meant to use the mouse position from Processing to control the LED and the buzzer in Arduino. When the mouse moves to a color, in other words, “lights a block”, the LED on the breadboard will present a color consistent with the color for the block. There are four blocks, three of them are for colors, red, blue and green and the last one is for the buzzer, when mouse is clicked on that box, the buzzer will be beeping.

Here is how the circuit looks like:

 

Here is how the circuit works:

 

How will I use serial Communication in my final project

My final project will mainly use “Arduino to Processing” serial communication. Since I will use some different sensors for the virtual experiments, I may use multiple data transmission which we learnt in the workshop. I think serial communication is very important for interaction design as it can combine physical part mainly controlled by Arduino and computer part featuring processing together. I hope I can apply what I learnt today well to my project!

Arduino To Processing [Arduino]---------------------------------------------
// IMA NYU Shanghai
// Interaction Lab

void setup() {
  Serial.begin(9600);
  pinMode(2, INPUT);
}

void loop() {
  int sensor1 = analogRead(A0)/4;
  int sensor2 = analogRead(A1)/4;
  int button = digitalRead(2);

  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(",");
  Serial.print(button);
  Serial.println(); // add linefeed after sending the last sensor value

  // too fast communication might cause some latency in Processing
  // this delay resolves the issue.
  delay(100);
}





Arduino To Processing [Processing]---------------------------------------------

/**
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */


import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 3;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ // This is the number of values you are receiving. 
int[] sensorValues;      /** this array stores values from Arduino **/

int xPos = width/2; 
int yPos = height/2;
int red = 255;
int green = 255;
int blue = 255;

void setup() {
  size(500, 500);
  background(0);

  setupSerial();
  ellipseMode(CENTER);
}


void draw() {
  background(255);
  updateSerial();
  printArray(sensorValues);
  
  xPos = int(map(sensorValues[0], 0, 255, 0, width));
  yPos = int(map(sensorValues[1], 0, 255, 0, height));
  
  fill(red, green, blue);
  stroke(0);
  strokeWeight(5);
  ellipse(xPos, yPos, 100, 100);
  
  if (sensorValues[2] == 1) {
  red = int(random(0, 255));
  green = int (random(0, 255));
  blue = int (random(0, 255));
  }

  // use the values like this!
  // sensorValues[0] 

  // add your code

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 3], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = 'n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}





Processing To Arduino [Arduino]---------------------------------------------

// IMA NYU Shanghai
// Interaction Lab

void setup() {
  Serial.begin(9600);
  pinMode(2, INPUT);
}

void loop() {
  int sensor1 = analogRead(A0)/4;
  int sensor2 = analogRead(A1)/4;
  int button = digitalRead(2);

  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(",");
  Serial.print(button);
  Serial.println(); // add linefeed after sending the last sensor value

  // too fast communication might cause some latency in Processing
  // this delay resolves the issue.
  delay(100);
}



Processing To Arduino [Processing]---------------------------------------------



/**
 * This example is to send multiple values from Processing to Arduino.
 * You can find the arduino example file in the same folder which works with this Processing file.
 * Please note that the echoSerialData function asks Arduino to send the data saved in the values array
 * to check if it is receiving the correct bytes.
 **/


import processing.serial.*;

int NUM_OF_VALUES = 4;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


float r = 255;
float g = 255;
float b = 255;
int s = 0;

Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(400,255);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[3], 9600);
  // check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index of the port

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;
}


void draw() {
  background(0);
  if( mouseX < 100){
    r = map(mouseY, 0, 255, 255, 0);
  }
  if( mouseX > 100 && mouseX < 200){
    g = map(mouseY, 0, 255, 255, 0);
  }
  if( mouseX > 200 && mouseX < 300){
    b = map(mouseY, 0, 255, 255, 0);
  } 
  fill(r,0,0);
  rect(0,0,100,255);
  fill(0,g,0);
  rect(100,0,100,255);
  fill(0,0,b);
  rect(200,0,100,255);
  if (mouseX > 300 && mousePressed == true){
    s = 1;
    fill(0);
  }else{
    s = 0;
    fill(255);
  }
  rect(300,0,100,255);

  // changes the values
  
  values[0] = int(r);
  values[1] = int(g);
  values[2] = int(b);
  values[3] = int(s);

  // sends the values to Arduino.
  sendSerialData();

  // This causess the communication to become slow and unstable.
  // You might want to comment this out when everything is ready.
  // The parameter 20 is the frequency of echoing. 
  // The higher this number, the slower the program will be
  // but the higher this number, the more stable it will be.
  echoSerialData(20);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    //if i is less than the index number of the last element in the values array
    if (i < values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    //if it is the last element in the values array
    else {
      data += "n"; // add the end of data character "n"
    }
  }
  //write to Arduino
  myPort.write(data);
}


void echoSerialData(int frequency) {
  //write character 'e' at the given frequency
  //to request Arduino to send back the values array
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {
    //add on all the characters received from the Arduino to the incomingBytes string
    incomingBytes += char(myPort.read());
  }
  //print what Arduino sent back to Processing
  print( incomingBytes );
}

Ix Lab Recitation 8: Digital Fabrication (Antonius)

Name of the recitation: Digital Fabrication

Professor: Professor Antonius

Partner for partial step: Cecilia Cai

 

GOALS FOR THIS RECITATION:

-Use methods of digital fabrication to realize and construct models.

-Use software or website like Illustrator and Tinkercad to prepare designed files.

-Actualize and digitally fabricate what we designed.

-Document the work and add a reflection.

-❤️

 

Designs

Cecilia and I decide to build a 3D model because neither of us have any experience in 3D design and let alone 3D fabrication. Both of us have used Photoshop or Processing to conduct some 2D designs before but when it comes to 3D design, we are completely “green hands”. Though Tinkercad is considered to be a basic beginner-level 3D design tool, we still find it challenging to build an exact model as what we expected. Because for 2D design, you only have to focus on four directions: left, right, up, down and the size of the object will not change according to its different positions, but in 3D world, there are more directions to be considered and the shape of the object will “change” when you see it from different angles.

For example, as you can see in these two images, the position of “Chic Rabbit” seems to be perfect from the angle of the first image but gets terribly wrong from the perspective of the second one. Thus, the lesson we learnt is that for 3D model building, we have to check all the angles in order to make a good design like what we imagined in mind.

 


(Sorry that there is a little typing mistake in the image)

The inspiration for our design comes from two components in the category of “Characters” in Tinkercad–“Chick foot” and “Bunny ear”. Cecilia first wanted to make a bunny but I thought it would be more interesting and special if we could make a unique (weird) rabbit which has chick foot. “Chick” sounds similar to  “Chic” so we decided to name our rabbit “Chic Rabbit”, which sounds like a name for a cool fashion brand. For the first rabbit, we tried to make it look cool and cute just like a popular figure of the fashion brand Moschino:

(cr: link)

So here is how our first “Chic Rabbit” looks from several different angles:

 

In oder to make our “fashion brand” more all-round and appeal to more age groups, we then create another figure which is named “Baby Chick Rabbit”:

 

We decide to make them colorful as what is shown in the images, however, we are not gonna to make them colored while 3D printing them. Instead, we decide to only use white material to fabricate these two models and then use pigment to color them with paint brush. 

 

Fabrication

Thanks for Professor Rudi’s advice, we realize that it may be impossible to print the whole rabbit at one time with the 3D printer because the structure of the rabbit is a little bit complex and it may be not stable when it is being printed. 3D printer works as overlying the material from the bottom to the top, and since the bottom part of our model is two chick legs, which are smaller than the middle part of the rabbit’s body, partial upper part won’t have enough support underneath, and the printer won’t be able to print out the shape successfully. So he suggests that we should break the rabbit into several parts and assemble them together afterwards. For example, first print out their chicken feet separately and then print out their bodies. The texts on their body may also need to be individually printed. Thus, it may be a long process of waiting.

For the fabrication, we decide to first print the baby chic rabbit because it is relatively less complex and may be easier to succeed. We first dissemble it into three parts–“a hand”, an ice-cream and the main body.

The process is long but magical!

 

This is what we get from the machine and there are many supporting parts that have to be eliminated later.

 

After Cecilia put them together and tried to remove all the unnecessary parts, the final look of our chick rabbit looks like this:

 

I think in general, it is quite a success and it turns out that the method of “first dissembling and then assembling” is a really helpful manner. It seems that the chick feet also need to have been printed separately because now those two feet are blocked by the supporting part.

 

Reflection
  • Just as what I mentioned before in the section “DESIGNS”, we decided to use Tinkercad to make our 3D models because first of all, both of us have no experience in 3D fabrication before and are eager to make some new attempts in new areas. Second, since we want to make some cute and cool characters, I think 3D is more suitable as they can make our characters more real-to-life and vivid. Last but not least, both of us are greatly interested in Tinkercad itself because as a open-source website, it provides creators with many basic geometric shapes such as the chick feet and bunny ears which provides us with great inspiration. My love and respect for this kind of open-source platform is also a driving force for my choice.

 

  • My Stupid Pet Trick–“Peppa Pig’s Papa Prank”, which is a stuffed toy of Peppa pig’s papa whose arms will move at a speed depending on the distance between the user and its nose. I didn’t use a lot of crafting methods in my project because the stuffed toy was bought from toy shop and the only thing I “did” to him was (a little violent) to replace his original “arms” with two cardboard “artificial limbs” which were lighter so that the servo could motivate them more easily and accurately. Though the final result turned out to be quite successful, I do think that it could be better if the pig’s arms are not “artificial but its original fluffy ones. However, from my failed attempts, I found out that if I want to use my servos effectively motivate its original arms which are quite heavy, I have to make a specially designed connection part to fix the servo tightly to its arms. The connection part I need has to be custom-made because there are no such things that I can find to serve this job and this is also one of the biggest reasons explaining why I gave up this plan in the end. However, if I had learnt crafting method I used in this recitation before, I would not have to give up my original plan because by using 3D printing, I can easily fabricate the exact shape I want to make. I think one of the biggest strengths of 3D printing as well as laser cutting is that they can actualize those designs which otherwise can only exist in our imagination. With the help of this kind of crafting method, we can make some really complex models depending on our anticipation and need. It is like that we can make custom-made good by ourselves and we can realize our design without worrying about where to find the components we want. This technology can no doubt open up more possibilities for people’s creativity and physical fabrication. When it comes to my midterm project, though I mainly focused on Processing part, I also designed an “interaction box” inside which I attached a potentiometer and an arduino along with a breadboard. The box was not perfect for my project because it was only the best one in the cardboard room and the size of it is only relatively suitable. The breadboard cannot be fit into the box and has to be attached outside which is not beautiful neither. So same with the Stupid Pet Trick project, the crafting method I used in this recitation can create a more personalized, more suitable box for my project and can make it also look better. For my final project “A Mini Chemistry Lab”, I may use the 3D printing to make some experiment equipment so that my sensors can easily be fit into those equipments. And also more in a more beautiful manner.

 

  • The reading “The Digital Fabrication Revolution” mainly discusses the revolution of widespread application of controlled machines in commercial field and analyzes the impact of some fabrication methods controlled by computers such as 3D printing which is the crafting method I used for my recitation project. An important argument mentioned by Gershenfeld is that computer-controlled fabrication is “personalization, producing products for a market of one person” (Gershenfeld 47) which suggests that computer fabrication is usually specially designed for a particular usage and is therefore unique and flexible. This method is consistent with my idea about 3D printing as I also see it as a way to personalize our creations and actualize our designs. Computer-controlled fabrication makes it possible for not only companies to produce costom-built products but also for creators to enhance their own creative projects.

 

  • In 2149, I guess that there will be mainly three changes for digital fabrication: speed, size and multiple materials or colors. First, the process of 3D printing will become much faster than now, several hours may become several seconds. Second, IMA student may get the chance to make some bigger 3D printing projects such as bags or chairs. Third, now it is hard for a 3D printing machine to make something that involves multiple materials or various kinds of colors due to the technique limit. But this problem may be solved in the future so that students can make more complex design and do not have to color their projects after printing like what Cecilia and I plan to do. But I think that the basic method of printing may not be changed–to print one floor after one floor to finally assemble as a project. Also, the materials that we can use today will also be available in 2149.

 

Ix Lab Final Project Essay by Emily Qiu (Antonius)

What is interaction? For me, the most accurate synonym for this concept may be the word “communication”. I think interaction is a mutual communication between different objects but with a specific emphasis on the effect of that communication. In other words, interaction is not only A does something to B but also B has to do something back and this exchange of behaviors must create something–either something visible or merely a invisible feeling. Interaction is not a one-way causal effect but rather, the connection established between objects can actually influence each other correspondingly. In a specific case, interactive art is like a cooperated work–the user does one part and the art installation does the other, they play different roles in the interaction but ultimately create something together. In addition, I think most interactions should be designed in a people-centered manner because despite that the non-human part is also an indispensable component of the interaction, the leading role is often grasped in the hand of the users. User’s experience is vital for design because they are more like the active operators in this communication.

VR is in my opinion an important interaction as it involves a mutual communication between the user’s motion and the virtual reality on the screen. They have influence on each other and in essence, they are creating something– a unique experience of the user and maybe some virtual creations in that virtual reality! The user plays an active part in this communication and the non-human part supports it with digital response, which is a sort of cooperation with a user-centered focus. VR is also like a interactive bridge between virtuality and reality where people are interacting with both the physical world by making motions and the virtual world by seeing response in the screen.

The project Grace and I want to propose is “Mini Chemistry Lab”. It is an educational as well as entertaining project which aims to help children to explore the world of chemical experiments in a easy and safe manner. The project will basically have two parts–the physical part which mainlly utilizes  Arduino and the Processing part. It is a little bit like a VR chemistry lab, which is also one of our references, but it is different from that as our user can actually operate on some touchable objects with sensors built in like beakers, muddlers and so on. The process of the experiment will be displayed on the screen depending on the user’s interaction and the user can therefore experience some interesting and really dangerous experiments safely. Our inspiration for this project derives from my personal experience when I was a child that I was very curious about the chemistry world but teachers didn’t allow us to explore it as it could be very dangerous. Children always make mistakes and some chemicals can be really toxic. Therefore, just as how I defined interaction as a people-centered mutual communication, our project will be designed totally “for our users”. The interaction we designed will be completely based on the need of a children “chemist”, for example, the interface will be colorful and there will be warning or even “explosion” on the screen when something goes wrong to serve as an educational reminder. It also corresponds with my definition of interaction as there are various mutual influences between my user and the project and there is something created in this process– the user can acquire chemical knowledge and they can also create some amazing “chemical effect”. It is like a cooperated work where children are engaged in the virtual experiments using their curiosity and the computer will respond to them, which ultimately create something new. The VR style of interaction is of great significance to our initiation because it helps us to realize that the theory of interacting with both virtuality and reality can also be applied to interaction design.

Ix Lab Recitation 7: Drawing Machines (Antonius)

Name of the recitation: Drawing Machines

Professor: Professor Antonius

Partner for partial step: Cecilia Cai

 

GOALS FOR THIS RECITATION:

-Learn to build a circuit to control the stepper.

-Use potentiometer to control the motor.

-Build a mechanical arm and combine it with another person’s to build a drawing machine.

-Document the work and add a reflection about the use of actuators, the digital manipulation of art, and the creative process.

-Choose an art installation mentioned in the reading ART + Science NOW, Stephen Wilson (Kinetics chapter) and post thoughts about it.

-❤️

 

PART a / STEP1: build a circuit to control the stepper

Components (credit to http://ima.nyu.sh/interaction-lab/category/recitations)

  • 1 42STH33-0404AC stepper motor
  • 1 SN75440NE ic chip
  • 1 power jack
  • 12 VDC power supply
  • 1 potentiometer (from your kit)
  • 1 Arduino and USB cable (from your kit)
  • Laser-cut and 3D printed mechanisms
  • Pen that fits the laser-cut mechanisms
  • Paper

 

This step seemed to be relatively easy because the only thing we had to do was to assemble our circuit according to this explicit guidance image:

(image credit to http://ima.nyu.sh/interaction-lab/category/recitations )

However, it turned out to be quite challenging because I was so afraid of “exploding” my laptop that I had to paid even “excessive” attention to my circuit 😂… I tried to make every wire, every pin and even every color consistent with the instructions and double-checked the circuit with Cecilia. I finally got this circuit and fortunately, it worked well and my laptop was safe and sound ❤️:

 

The motor is moving smoothly!!!

 

PART a / STEP2: Use potentiometer and the MotorKnob example to control the motor

The example sketch was actually very complete itself and the only issue was that 42STH33-0404AC stepper motor is a 200 step motor and I had to use map() function to match the movement of the knob with the rotation of the motor. So I just set up another variable “val1”, as you can see in the screenshot before to make my map() function more clear and explicit. The key line is : int val = map(val1, 0, 1023, 0, 200). Here I suddenly realized that map() is such an important function that it worked like a adapter or translator which made it possible for different agencies to communicate with each other.

Here is my work for this step:

 

PART a / STEP3: Assemble the drawing machine

This step was a lot of fun because we could actually create something with the tool we created! It was also a very creative way of drawing and I thought it was really interesting! The process of exploring which position of the two motors can create a specific kind of patterns is really fun and I could feel  a strong sense of achievement. Here is how Cecilia’s and my machine looks like:

 

Here is video clip showing how it actually works:

 

Though it was really hard for us to control the machine accurately, we found it really interesting to explore the “rules” of how it worked. The process of drawing required “a high level of corporation”, so you could always hear how Cecilia and I “screamed and yawned” to “communicate with each other” in order to get a line we expected — though the result was always a failure. the line would always went crazy in the direction of abstraction. Drawing by hands is already a difficult thing for me to do, but it is much easier compared with drawing with my drawing machine. We devoted a lot of effort to our creations and here are two final “successful” ones:

 

 

 

We also tried to experiment with changing speed and you can see the result on the right side of this drawing

“crazy fairy”. I know it is quite abstract but…it is “a great work for Modernism”  you know……Ireally love this drawing!

 

REFLECTION FOR MY DRAWING MACHINE

When I was little, I always regarded machines that can move automatically themselves are nothing but magic! I couldn’t work out the reason why things without lives could also move in a certain manner as if they were alive. Now, when I grow up, when I take this Ix Lab course, when I finally finish my first machine with motors, I think I may know at least the partial secret behind the “magic” —actuators. Actuators are very useful for that they are able to not only move themselves but also motivate other objects to move with them. For example, in this exercise, the motor can motivate the 3D-printed arms attached to it and more importantly, to let it move in accordance with our instructions. I think the combination of actuators and coding is such a big progress that it makes it possible to flexibly change the function of the actuators. The way how the actuators work can be easily changed by the new codes imported, so the actuators are not limited to only one use but equipped with a variety of possibilities. For example, for my drawing machine, I may use a set of codes while next time, if I want to make maybe a cookie machine, I can use the same actuator with some new adaptations to its codes. Thus, in my opinion, the combination of physical devices and coding  is actually a great progress for actuators.

When it comes to the digital manipulation of art, though it has some disadvantages, for example, it is sometimes hard for artist to actualize their ideas because human’s logic is not digital and machines and brains are two completely different systems, this new form of creating art has many unique strength: digital element can offer more possibilities for artists and make art more accurate when needed. Digital manipulation can make art with accurate data which is especially useful when the art work needs to be mass produced.

Also,  abstract work is really interesting because we need to use our “wildest” imagination to create and appreciate. It becomes more interesting when sometimes different people have different ideas about one drawing as differences are the reason why art and this world so diverse and attractive. For the drawing “Crazy Fairy”, I named its fairy while one of my friend saw it as a Christmas tree? I think it is really a interesting process and maybe it should be the charm of art!

I remember that since when I was little, I have always dreamed to make a sticker-making machine because I was crazy about collecting stickers then. I would like to make a machine that had two parts, one is for me to draw or design what kind of pattern do I want while the other part can actually transform my design into real stickers. I think it is very meaningful because it has combined design and actual objects together and I can therefore print out some stickers that I want to make myself!

 

Thoughts on “ART+Science NOW”

The art installation I choose is Waves by Daniel Palacios Jimenez.

I like this work because despite its simple outlook, it can create multiple effects–both complex sound and visual wave oscillations. I like its design that only uses a elastic rope to perform complex motions as it gives users a magical feeling–looks simple but has infinite power. I also love its interaction design that the rope will gives corresponding reactions to users’ motions: stillness means no motion, simple motion produces regular sine waves and much motion gives chaotic patterns and sounds. This interaction is both easy for users to explore on their own and interesting for them to get involved in. I think the process of finding the borderline of “simple motion” and “much motion” is probably one of the most fun part of this interaction because the user will feel surprised and curious when they find the difference. Also, using motors to rotate is also a good design point because rotation can always give people a feeling of infinity and limitlessness.

I think both this art work and my drawing machine have a similar characteristic that they both give instant feedback to their users. The elastic rope will perform different patterns of waves depending on users’ motions while the drawing machine will “draw” lines based on the orders given by users. No matter the waves or lines, they all give users “a feeling of manipulation” which makes them feel that they are playing a role in the interaction. But I feel like the Waves is easier for users to interact with as it only requires users to do some motions while the drawing machine needs users to switch the potentiometer which is really hard to accurately control the pen. However, I think the drawing machine has a very good concept because it needs two people to corporate. In my opinion, the element of collaboration is a good point to add in creating an interactive art because it is cool if one art can connect two people who are even completely strangers together to make something. I always believe that people are very important for art design and it is one of the charm of art.

I guess the artist choose those specific motors as the actuators for his project depending on two aspects. One is the practical use, in other words, he needs something that can actualize his design. So for this project, he has to use some actuator that can rotate the elastic rope in different ways depending on the orders it receives. The other aspect is probably the consideration for outlook design because I think the outlook of the project is also very important. Besides the function of the motor, it is also important to make the whole project look nice, so I guess the artist also tries to find some actuators small or “invisible” in the project.

#include <Stepper.h>

// change this to the number of steps on your motor
//-----------------------------------------------
#define STEPS 200

// create an instance of the stepper class, specifying
// the number of steps of the motor and the pins it's
// attached to
Stepper stepper(STEPS, 8, 9, 10, 11);

// the previous reading from the analog input
int previous = 0;

void setup() {
  // set the speed of the motor to 30 RPMs
  stepper.setSpeed(30);
}

void loop() {
  // get the sensor value
  int val1 = analogRead(0);
 //-----------------------------------------------
  int val = map(val1, 0, 1023, 0, 200);

  // move a number of steps equal to the change in the
  // sensor reading
  stepper.step(val - previous);

  // remember the previous value of the sensor
  previous = val;
}

Ix Lab Recitation: User Testing (Antonius)

Name of the recitation: Guest Talk

Professor: Professor Antonius

Me: Emily Qiu

 

Part 1: Prepare
  • A list of things that I want to know when my users interact with my project:
  1. In general, how do they feel about my project?
  2. Which part or element that attracts them the most?
  3. Where do they think that needs further improvement?
  4. What other suggestions do they have?
  5. Do they think that the project is useful or meaningful?
  6. What do they think about the interaction design? The interface? The final result?
  7. …………😊

 

Part 2: Testing

A) Testing my project

 

These two video clips show that how Cecilia tested my project. Actually many people had tried my project but they were a little too “shy” to be filmed 😊…

 

B) Being a user

This video shows that I was a user for Cecilia’s project–Eat the Beans.

 

Part 3: Reporting
  • A project I tested: “Eat The Beans” by Cecilia

It is an interesting game that user can press button to start or reset and press  up,donw, left, right key to manipulate the little creature. The goal is to let the little creature eat only purple particles and if he got collided into a blue one, he died. There are several levels and the little creature will continue to be bigger if he eats some “beans”.

I found this game really interesting. I was not familiar with this game at the beginning, so I didn’t do very well. And also, because there were no instructions on the screen or somewhere to tell me what to do, the only thing I knew was to press the start button but then I didn’t know what to do next. I didn’t know that the size of the creature was decided by how long you pressed the button, so I immediately “died” at the first time. Then Cecilia told be the rules and I gradually became good at playing this game. The game was also a little bit tricky because the position of the beans was randomly decided, sometimes it was nearly impossible to eat the bean without touching those blue particles but sometimes it was easy. In the end, I won the game by getting ten points in total and it was great fun.

Some questions and suggestions I came up with:

  1. Put some instructions either on the screen or merely on a piece of handout. Because it is hard for people to understand the rules and what to do if they don’t get enough instructions or guidances.
  2. Avoid “suicide” condition. Sometimes when I started the game, the creature would die immediately because the original position of the creature was on a blue particle. Though it was also kind of interesting and ironical, I think by doing some adjustment to the setting of the distance maybe, this kind of condition can be avoided?
  3. Maybe add some music or sound to the game. I think since it is a fun game, adding some sound effect or background music should be a good choice to be considered.

 

  • My own project: “Drawing by Spelling” 

There are two typical ways how my users interacted with my project:

One is like Cecilia, they were not familiar with the instructions at the beginning but could gradually got to know them and could enjoy the game in the end. Some people said that they really liked the project and wrote some really cute comments on my notebook and I really love them! The other is that they don’t have enough time or “patience” to read my instructions and the bell rang so fast, so they couldn’t finish the whole game. I think if given enough time, the result could be so much better. Generally, when they were changing the color of the balls or flower, or especially when they finished the whole game and used the “torch” to see their creation, many of my test users let out a “wow” and I felt really happy about it.

When it comes to my users’ comments, I summarized them as below and I have also added some reasons behind them and some potential solutions or improvements.

  1. “Maybe to make the instructions more clear and simple”: I think it is really a good suggestion because I was also struggling when I set those rules. I made many detailed rules because I was afraid that the user wouldn’t understand my intention. But it turned out to be that many users wouldn’t have the “patience” to read the whole instructions before they really started the game. Many of them just liked to ask “How can I play this game? What should I do?” without really reading the instructions. Thus, my improvement was that I would try to simplify the instructions to merely some key words, just like what Professor Rudi told me. I think it would be of great help to my future projects.
  2. “Add some sound effect or background music”: This is also a suggestion I gave Cecilia and I think it is a very constructive idea. I also thought about adding background music but I gave up the idea later. Now I have seen many successful projects with sound, I have asked them about the library they used and I will try them next time!
  3. “Add more diverse interaction”: I’d love to do this if I have more time and more techniques and I actually decide to keep improving this project as I am learning more things.
  4. “Interesting and nice intention”: I really like this comment because I have really spent a lot of time thinking about the meaning for my project as you can see in my previous documentation. I do think that a good project  will somehow help some certain people and I want to make one like that 😊!
  5. “Cute interface but maybe use “delete” to replace “.”: Just as what I have explained in my midterm documentation, I also wanted to use backspace or delete, but failed somehow. I will continue working on this!
  6. “I like it in general but switching while pressing, a little bit busy for me”: This is a comment from Cici, who was my instructor for Comm Lab. I think it is really a good suggestion for me because I also find it a little struggling to do both two things at the same time. However, I didn’t know how to differentiate each function if I don’t use different keys to distinguish. But Cici gave me some really good suggestions such as getting rid of keyboard but to use buttons, then all interactions will be put on the box and button don’t have to be pressed whole time. 

 

Ix Lab Midterm: “Drawing by Spelling” (Antonius)

Name of the Project: Drawing by Spelling

Professor: Professor Antonius

Producer: Emily Qiu

 

title and subtitle

The title of my project is called “Drawing by Spelling”. I tried to make my title both concise and explicit but it was much more harder than what I had expected. To summarize is really a difficult yet important skill that I really need to work on in the future. When it comes to the subtitle, I named it “an amazing educational tool for kids” because this subtitle made it clear that my project serves an educational aim and the target users are kids. As for the word “amazing”…I just wanted to make it sound more attractive…😂…

 

Description of the project

This project is an educational tool that is aimed to help kids who are not native English speakers to study English words. It is basically a rewarding system to test kid’s spelling ability and give them some treats as motivation if they type the word correctly. The project has mainly two parts, an interaction box with a potentiometer and a interface in processing. So basically, user will be asked to type in the English word for some images following a certain order and if the answer is correct, he/she will be taken to another page to do some interactions–“drawing”, by switching the potentiometer, to change the color, shape or other characteristics of the objects in the page to create their own, unique image. They will be asked to save the image and in the end, they will be able to get a complete drawing and see it through a “torch”. To summarize, the project is meant to bring children a sense of achievement to encourage them to spell English words in a fun way.

(homepage for the interface of my project)

 

demo

(The complete file is too big to be uploaded so I have broken the video into two parts)

(In case of some incidents, here is a link to my HD demo which is in my google drive: https://drive.google.com/open?id=1UEAl4e5WDfIArRzVjTQORcskCphrRDkC)

 

There is something confusing and unclear in the demo because it was made for my presentation and in my presentation, I have designed  to give some comments while playing this demo. So if there is something hard to understand in the demo, I will further explain how my project works in the TECHNICAL DEVELOPMENT part. But I hope that this demo can give you a general sense of my project.

 

Conceptual Development — Ideation
  • #Inspiration# My inspiration mainly comes from my younger brother–Aaron. He is a four-year-old little naughty boy and he just hates spelling words. He would rather eat eggplants (which he hates so much) than study English words. But one day, everything changed when he got a gift from my uncle. It was an educational toy which helped kids to develop interest in spelling words. There are a bunch of blocks and you can put them into some blanks. The physical blanks are connected to an app on iPad and if the kid type a correct word, there will emerge accordingly some cartoons on the screen. Aaron was so attracted to this game that he almost spared no effort to spell words every day. Thus, I think it might be a good idea to use this method to build an interactive educational tool. 

(photos credit to my mom ❤️)

 

  •  #Motivation# My motivation comes from my personal experience and this can be well explained by a series of questions I put forward in my presentation: “How many of you are not native English speakers? So have you ever felt that spelling those english words can be so boring? Especially when you were a kid, and like me, I used to think that those words are even much more terrible than spiders. So I was wondering, can spelling words also become a cool thing, a fun thing to do?” Though there are some tools like the one my brother had, I think it would be more interesting if I can combine spelling and drawing, or more accurately, creating together. So this is my initial motivation.

 

  • #Brainstorming# Since I have already decided my topic so my brainstorming is mainly focused on what words to choose and what interactions to build for each one: (sorry for having a terrible handwriting and draft 😂)

Conceptual Development — Research
  • #References#  
  1. PBSkids organization/ spelling games: this is a website full of different kinds of spelling games and I have learnt a lot from them such as interface design: http://pbskids.org/games/spelling
  2. Processing organization: when I was thinking about the interaction design, I refer to this website for help:https://processing.org
  3. Pinterest: my images and some visual elements are credited to this website: https://www.pinterest.com

 

  • #Target Audience# As what I have mentioned before, my target audience are kids ranging from 5-12 (or elder) who are not native English speakers or who are but have no interest in learning spelling. But I think that maybe this theory can also apply to other language learning such as Chinese, for example, maybe when you write a character correctly you will get a reward. But maybe for Chinese, a keyboard is far from enough, maybe we will need a tablet.

 

  • #Needs Analysis# Through my research, though there are many spelling games, I don’t see any games that combine the concept of creating something with spelling words together yet for me, I think it is an interesting and promising idea. If I have more time and skills to build a bigger project that have more words, then the user can in the end get a more fantastic “drawing”! I think there is a need for this kind of project.

 

technical development–design
  • #Flow Chart# 

 

  • #Design# Given my target audience and my needs analysis, I have mainly applied three principles in my design.
  1. Visual attraction. To visualize everything. From putting image as a question instead of a translation, it is better for kids to remember that word better. Kids love colors and drawing, especially in a dynamic, digital manner.
  2. Rewarding system. It is inspired by my mother. I have a four-year-old brother. My mom says that one word he can spell that day equals to one minute for television and this does work. Kids need rewards as a motivation. So I have designed that a correct answer means an access to each step and kids can finally create something and can save them as files.
  3. Interest matters. When I first ask feedback on my initial idea, some argues that it is too deficient that kids can only remember few words a day. However, I think what matters most is the interest and the quality of his memory instead of only quantity. Interest is really important for education.

 

technical development–Implementation & lessons learned
  • #Physical Part# #Process#

Initially, I just wanted to use the keyboard as an interaction tool because it seems to be the simplest way to type in. However, then I thought it might be a little boring to just interact with the keyboard. So I was wondering if I could combine Arduino and Processing together to create something like an “interaction box”. I also knew that the interaction design shouldn’t be too complicated to be practically used, so I prepared to move all my interaction to that box. I thought maybe I could make some cards with some special sensors and can be somehow tested whether they are in a certain order (the correct word) or not.

However, later when I was actually making those “special cards”, I found that there were no suitable sensors for them. I attempted to use some distance sensors to test the distance between two cards in order to know if their position is right or not. But later I found this method impractical because distance doesn’t have a direction, if the correct order is “CAT” then “TAC” also can be tested to be right. And the sensor would also easily be distracted by other objects aside from the cards, for example, when user was moving those cards around, the distance sensor might test the distance between the hand and the cards. Also, it was too hard to distinguish each card as the sensor cannot specifically, merely test the distance between two specific objects. Ultimately, after a series of attempt, I gave up this idea and turned back to my keyboard 😭😭.

I was then inspired by an exercise I have done before which I used potentiometer to change the opacity of the objects on processing (🙈 but later it turns out that my so-called  “self-improvement” is a little problematic for many users). So I decide to combine keyboard and potentiometer together— use keyboard to type while use potentiometer to control/interact. The result you can see in the demo.

(the “interaction box”, I know he is a little ugly but it is “environment-friendly” 😂)

  • #Physical Part# #Lesson Learned#
  1. Simplify the interaction platform. From the feedback I get from the user testing, there is mainly one big issue of my physical interaction design — sometimes a little bit too complicated. Just as what Professor Cossovich suggested, it is a little too hard for user to focus on two places at the same time, so it can be better if you only use the interaction box or the keyboard. I think this suggestion is of great value to my future design.
  2. Beautify the box. The outside decoration is actually very important and if I have more time and materials, I will definitely to try to build a better one!

 

  • #Coding# #Process# (the complete sketch is in the end)

The process of coding is in fact very very tough because I admit that my logic is really different from the one the computer can understand. Thus, sometimes a small step, like how to present the word the user is typing  or delete the wrong word, can be really difficult for me to resolve. However, it is also great fun working out all those problems and encountering a fantastic final result! Thank for all the help I got and I really learnt a lot from this project. Here are some parts of the codes that impressed me and taught me the most:

1. How to let user type words?

It was the first problem I met during my coding and since I knew nothing about typing text in processing before, I turned to processing references for help but failed. I didn’t find anything about typing input except some knowledge about how to use strings and text from this page https://processing.org/tutorials/text. Then I tried to google some examples about type in processing and I got this source http://www.cs.du.edu/~malbow/COMP1671/Notes/ProcessingNotes7.pdf . It was really helpful for introducing strings and input/output.  

In regard to how to delete the wrong answer, I have learnt something from another example (backspace to delete).

After carefully exploring these examples and codes, I then wrote something for my project. I have changed something such as press ‘.’ to delete instead of pressing backspace because I don’t know why if I use backspace, the code wouldn’t work and I had to use a not-special key instead. I know it is not convenient and I have to later figure out the problem.

I have kept the line “backspace” but I still need to work on in to let it function well. For this part, I have also utilized “null” and “str()”.

 

2. How to test user’s answer and change page?

I thought it was a easy piece…but…in fact, I spent a lot of time working on this part. The problem was that I didn’t know how to write the statement for my if condition. I have tried to write “if (a==”ball”)” and ” string d=”ball”…if (a==d)”, but they both didn’t work. I felt so confused and I turned to the processing website again. After another series of failure, I found the function “equals()” and this time I got it! The final statement should be “a.equals(“ball”) == true”.

When it comes to the switch of different pages, I set an variable “mode” and let mode respectively equal to 0,1,2,3, and then put every page’s codes under the condition “if (mode==1)” or else.

 

3. Other

Reference for the following: https://processing.org

I have used serial communication to connect Arduino and Processing together. For bouncing ball and torch I have learnt something from the examples in processing website. For importing images, I mainly used the function “image()”; For text’s font, I mainly used the structure of  “font = loadFont(“ComicSansMS-Bold-48.vlw”); textFont(font1, 25);”…… To summarize, I have really learnt a lot from this project!

(Cecilia’s work, one of my test user,by using my code)

 

  • #Coding# #Lesson Learned#
  1. Combine different examples together. When I don’t know what to do and I have no experience before, it is important to go to processing website or elsewhere to look for examples or references. But it is not right to just use them as they are but to create something new for my own project. Sometimes different examples will have different advantages and it is fun to combine them together.
  2. Failure is not something shameful but precious sometimes. There were indeed a lot of failures in this project and I used to be a kind of person who is very afraid of making mistakes. However, from this project, I got to know that sometimes small failures are just paving the way for bigger success.
  3. To make the coding organized is a necessity. This time I have nearly four hundred lines of codes and it is of great importance to make them clear and organized. I used some comments and long lines to distinguish each part and it is really useful for debugging.
  4. Break each function into steps. This is one of the techniques taught by my Professor Antonius. I think it really helpful to break a big, complex function into some small steps.
  5. Debugging needs patience. It is always annoying to meet things that don’t work but debugging is also a process of making critical progress.

 

  • #User testing#

This part will be detailed in the following documentation

Ix Lab Recitation: User Testing (Antonius)

 

Conclusion

Albeit there still exist a lot of problems and deficiencies for my project, I am quite satisfied with it because I think it is a quite complete composition and a good comma for the first half of this semester. I hope that I can learn more from this experience and make a better project for the final ❤️!

 

 

//Arduino```````````````````````````````````````````````````````````````````````````````

int sensorValue =0;
int outputValue=0;


void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);

}

void loop() {
  // put your main code here, to run repeatedly:
  sensorValue = analogRead(A0);
  int outputValue=map(sensorValue,0,1023,0,255);
  Serial.write(outputValue);
  
}


//Processing````````````````````````````````````````````````````````````````````````````

String a;
int mode = 0;
//balls
import processing.serial.*;
Serial myPort;
int valueFromArduino;

//Type words
PFont myFont;
String typedWord = "";
String correct1="ball";


//x:opacity,d:diameter
int r, g, b, o, d;
int r1,g1,b1,o1,d1;


//bouncing balls
int numBalls = 13;
float spring = 0.14;
float gravity = 0.11;
float friction = -0.9;
Ball[] balls = new Ball[numBalls];
PImage img;
int flag = 0;
int b2 = 0;
char nextkey;
int d2 = 0;




 //setup family-------------------------------------------------------------------------------------------------------------------------------
void setup() {
  a = "" ; 
  size(1280, 720) ;
  printArray(Serial.list());
  myPort= new Serial(this, Serial.list()[3], 9600);

  for (int i = 0; i < numBalls; i++) {
    balls[i] = new Ball(random(width), random(height), 100, i, balls);
  }
}




 //loop family---------------------------------------------------------------------------------------------------------------------------------
void draw() {
      while (myPort.available()>0) {
      valueFromArduino=myPort.read();
    }

  //test your vocabulary 
  if (mode == 0) {
    img = loadImage("bg.jpg");
    img.resize(1280, 720);
    background(img);
    fill(#4D463E);


PFont font;
// The font must be located in the sketch's 
// "data" directory to load successfully
font = loadFont("ComicSansMS-Bold-48.vlw");
textFont(font, 32);
text("What To Do?", 80, 315);
textSize(20);
text("#What is the English word for image 1 on your right side? ", 80, 370);
text("Type that word letter by letter (carefully and slowly :)", 100, 400);
text("#When you type the right word, something cool will happen!!!",80,430);
text("#When you somehow get back to this page,",80,460);
text("try the other two in order",100,490);
text("#type '.' to delete and the word will disappear",80,520);
text("if you are a little too wrong (*^__^*)",100,550);
fill(#E399C9);
text("*Remember:when you don't know what to do,",80,580);
text("Press the number key of the length of the word you just typed",100,610);


//Typing Part
    if (keyPressed) {
      if (keyCode == BACKSPACE) {
        a = a.substring(0, a.length()-1); //erases the last character,doesn't work really
      } else {
        nextkey = key;
        print(nextkey);
        if (a == null) {
          a = str(key);
        } else {
          a += key ;
        }
        delay(180);
      }
    }
    if (a != null) {
      
      textSize(30);
      fill(255);
      text(a, 150, 650);
      
        if (str(nextkey).equals(".") == true) {
        a= null;
        delay(150);
      } 
      
      if (a.equals("ball") == true) {
        print(12);
        mode = 1;
      }
      if (a.equals("flower") == true ) {
        mode = 2;
      }      
      if (a.equals("torch") == true ) {
        mode = 3;
      }
      
      
      
      if (a != null) {
        d2 = a.length();
      } else {
        d2 = 0;
      }
      if (d2 == 7) {
        text("", 20, 70);
        if (flag == 0) {
          b1 = millis();
          flag = 1;
        }
        int c = millis();
        int d1 = c - b2;
        if (d1 >= 2000) {
          a= "";
          flag = 0;
        }
      }
    }
  }




  //Mode 1 (ball) (Part1)---------------------------------------------------------------------------------------------------------------------------------
  if (mode == 1) {


    noStroke();
    img = loadImage("balls.jpg");
    img.resize(1280, 720);
    background(img);
    fill(r, g, b, o);
    for (Ball ball : balls) {
      ball.bang();
      ball.move();
      ball.show();
    }
  }
  
  
 //Mode 2 (flower)-------------------------------------------------------------------------------------------------------------------------------------- 
    if (mode == 2) {

 
noStroke();
    img = loadImage("myballs.jpg");
    img.resize(1280, 720);
    background(img);
fill(#FFD700);
ellipse(300,200,60,60);


fill(r1,g1,b1,o1);
ellipse(400,200,100,10+d1);
ellipse(200,200,100,10+d1);
ellipse(300,103,10+d1,100);
ellipse(300,300,10+d1,100);




pushMatrix();
translate(225,119);
rotate(radians(55));
ellipse(10,0,100,10+d1);
popMatrix();

pushMatrix();
translate(359,268);
rotate(radians(45));
ellipse(10,0,100,10+d1);
popMatrix();

pushMatrix();
translate(365,137);
rotate(radians(305));
ellipse(10,0,100,10+d1);
popMatrix();

pushMatrix();
translate(225,285);
rotate(radians(305));
ellipse(10,0,100,10+d1);
popMatrix();

if(keyPressed){
  if(key=='d'){
  d1=valueFromArduino;
  }else if(key=='o'){
    r1=240;
    g1=140;
    b1=195;
    o1=valueFromArduino;
  }else if(key =='r'){
    r1=int(random(0,255));
    g1=int(random(0,255));
    b1=int(random(0,255));
    o1=80;
    
  }else if (key=='s') {
        save("myflower.jpg");
      } else if (key=='q') {
        mode=0;
      }else if(key=='6'){
     PFont font1;
// The font must be located in the sketch's 
// "data" directory to load successfully
font1 = loadFont("ComicSansMS-Bold-48.vlw");
textFont(font1, 25);
fill(#B399E3);

        text("Secret Again!",340,220);
        text("Opacity:Switch the potentiometer while pressing the key 'o' :)",340,260);
fill(#6B0AC1);        
        text("Shape of your petals:Switch the potentiometer while pressing the key 'd' :)",340,300);
        text("Color:Press the key 'R' to get a random color for your petals)",340,340);
        text("Remember to press the key's' to save your drawing !!!",340,380);
fill(#B399E3);
        text("Press the key'q' to quit and go back",340,420);
      }
  
}
    }



 //Mode 3 (flashlight)--------------------------------------------------------------------------------------------------------------------------------------

  if (mode == 3) {
  img = loadImage("myflower.jpg");
  img.resize(1280, 720);
  img.loadPixels();
  // Only need to load the pixels[] array once, because we're only
  // manipulating pixels[] inside draw(), not drawing shapes.
  loadPixels();
      for (int x = 0; x < img.width; x++) {
    for (int y = 0; y < img.height; y++ ) {
      // Calculate the 1D location from a 2D grid
      int loc = x + y*img.width;
      // Get the R,G,B values from image
      float r,g,b;
      r = red (img.pixels[loc]);
      g = green (img.pixels[loc]);
      b = blue (img.pixels[loc]);
      // Calculate an amount to change brightness based on proximity to the mouse
      float maxdist = 50;//dist(0,0,width,height);
      float d = dist(x, y, mouseX, mouseY);
      float adjustbrightness = 255*(maxdist-d)/maxdist;
      r += adjustbrightness;
      g += adjustbrightness;
      b += adjustbrightness;
      // Constrain RGB to make sure they are within 0-255 color range
      r = constrain(r, 0, 255);
      g = constrain(g, 0, 255);
      b = constrain(b, 0, 255);
      // Make a new color and set pixel in the window
      color c = color(r, g, b);
      pixels[y*width + x] = c;
    }
  }
  updatePixels();
  }
  if(keyPressed){
  if(key=='q'){
  mode=0;
  }else if(key=='5'){
     PFont font2;
// The font must be located in the sketch's 
// "data" directory to load successfully
font2 = loadFont("ComicSansMS-Bold-48.vlw");
textFont(font2, 25);
fill(255);

        text("It's a torch!",340,220);
        text("Guess what is hidden in the blackness?)",340,260);
fill(#6B0AC1);
        text("It's your drawing! :)",340,300);
        text("Congratulations!!!",340,340);
fill(#B399E3);
        text("The file of your drawing is in the folder named 'myflower' :))",340,380);

      }
  }
}



 //Mode 1 (ball) Part2-----------------------------------------------------------------------------------------------------------------------------------
class Ball {

  float x, y;
  float diameter;
  float vx = 0;
  float vy = 0;
  int id;
  Ball[] others;

  Ball(float xin, float yin, float din, int idin, Ball[] oin) {
    x = xin;
    y = yin;
    diameter = din;
    id = idin;
    others = oin;
  } 

  void bang() {
    for (int i = id + 1; i < numBalls; i++) {
      float dx = others[i].x - x;
      float dy = others[i].y - y;
      float distance = sqrt(dx*dx + dy*dy);
      float minDist = others[i].diameter/2 + diameter/2;
     
      if (distance < minDist) { 
        float angle = atan2(dy, dx);
        float targetX = x + cos(angle) * minDist;
        float targetY = y + sin(angle) * minDist;
        float ax = (targetX - others[i].x) * spring;
        float ay = (targetY - others[i].y) * spring;
       
        vx -= ax;
        vy -= ay;
        others[i].vx += ax;
        others[i].vy += ay;
      }
    }
  }


  void move() {
    vy += gravity;
    x += vx;
    y += vy;
    if (x + diameter/2 > width) {
      x = width - diameter/2;
      vx *= friction;
    } else if (x - diameter/2 < 0) {
      x = diameter/2;
      vx *= friction;
    }
    if (y + diameter/2 > height) {
      y = height - diameter/2;
      vy *= friction;
    } else if (y - diameter/2 < 0) {
      y = diameter/2;
      vy *= friction;
    }


    if (keyPressed) {
      if (key=='o') {
        o=valueFromArduino;
      } else if (key=='r') {
        r=valueFromArduino;
      } else if (key =='g') {
        g=valueFromArduino;
      } else if (key=='b') {
        b=valueFromArduino;
      } else if (key=='s') {
        save("myballs.jpg");
      } else if (key=='q') {
        mode=0;
      }else if(key=='4'){
     PFont font3;
// The font must be located in the sketch's 
// "data" directory to load successfully
font3 = loadFont("ComicSansMS-Bold-48.vlw");
textFont(font3, 25);
fill(#B399E3);
        text("Secret!",340,220);
        text("Opacity:Switch the potentiometer while pressing the key 'o' :)",340,260);
        text("Color:Switch the potentiometer while pressing the key 'r' or 'g' or 'b')",340,300);
fill(#6B0AC1);
        text("Remember to press the key's' to save your drawing !!!",340,340);
fill(#B399E3);
        text("Press the key'q' to quit and go back",340,380);
      }
    }
  }

  void show() {
    ellipse(x, y, diameter, diameter);
  }
}

Ix Lab Recitation 7: Guest Talk From Xinchejian (Antonius)

Name of the recitation: Guest Talk

Professor: Professor Antonius

Me: Emily Qiu

 

 

GOALS FOR THIS RECITATION:

-Listen to those amazing talks given by project creators form Xinchejian.

-Reflect on the talk.

-Get inspiration or feedback for my project.

-❤️

 

Guest Speakers:
  • Andy García, Urban farming from Domosfarm

(source credit to http://ima.nyu.sh/interaction-lab/field-trip-2/)

 

My reflection:

I have to say that I really enjoy this event and I think it is really meaningful. All projects presented today are so amazing that I am sure this is literally what I want to do in the future. Here are my reflections on each project.

 

  • Xinchejian

Alarcon first introduced the community-run hackerspace, Xinchejian and I was really surprised about what a hackerspace really is because not until yesterday did I realize that hacker was not merely those who wrote codes to break into other systems. Instead, they are some amazing people and I would rather call them “creators”. I think Xinchejian is a very meaningful place because it is somewhere that embraces every creative or potentially creative idea! I remember that he said Xinchejian is a perfect place for both confident people who have an idea and anxious people who have no idea. I really love this description because I always believe that inspiration is not something that will knocks you door and all you can do is wait, instead, it is something that you can encounter through your effort such as talking to people, doing research and even just going out and touching the world. Given that now I have basically no idea for my midterm project, I will go to its website tomorrow and check this list of interesting projects, I hope that I will get sparkled soon!(credit to http://xinchejian.com)

 

  • TOKY

I think this is probably one of the tools that I am most desperate for this year because it is really cool, convenient and useful 🤣🤣🤣. Besides, I especially like the initial intention behind his project–to make some educational tools for the promotion of new technology. He made this decision after a long research and realizing that there is a blank space for educational use. This actually suggests the first step of a successful project — to establish a meaningful goal for the use of your project. We of course can make something for entertainment and even just for nothing, however, I think a good intention will no doubt make a difference as the project is meant to help others or help the development of an industry. I also want to make something that is of certain benefits to others and this world.

I am also interested in his project itself so I have gone to his website after the event and I have seen several interesting projects via his invention. This Hamster Gym is my favorite and I really hope that I can make one one day (maybe by usingmy Arduino!).

(image credit to and full video on:http://tokylabs.com/hamster-gym/)

 

 

  • Urban Farming

The speaker for this project is very humorous and interesting. I was deeply impressed by the story behind his project — plant mints for his mojito (?). I feel like in essence, our daily life and our own need may be the biggest source for creativity. Become a careful observer in daily life!

 

  • Precious Plastic

This is my favorite project because it is both meaningful to this world and “more importantly”, beautiful…☺️💕……I think the issue of environment is becoming more and more apparent these days and it’s true that many people become more aware of the potential risk we are all confronted with. However, as what two speakers pointed out, many people just didn’t know what they could do to protect our environment. What the two speakers do is to make abandoned plastic into beautiful art design, to give rubbish a new life so that they will not be thrown away but well reserved. I really like their idea and I really want to participate in their workshops in person to feel the charm of making plastic arts. The combination of recycling and arts is really creative and I really like it. 

I also went to their website and asked for their WeChat official account so that I can make an appointment with them next time. Below is one of my favorite work and I hope that I can make one in their next workshop. 😊😊

(image credit to: http://www.smartshanghai.com/wire/activities/precious-plastic-workshop)

 

Feedback for my project:

I feel pity that I didn’t have the chance to talk to them about my initial ideas due to the time constraint. But I am greatly inspired and I feel confident that I can make something cool this time! 😺😼😺

 

 

Ix Lab Recitation 6: Serial Communication (Antonius)

Name of the recitation: Serial Communication

Professor: Professor Antonius

Producer: Emily Qiu

 

GOALS FOR THIS RECITATION:

-Create a Processing sketch that sends data to Arduino. Receive that data and express it using physical components (ie. servo, led, buzzer).

-Create a circuit using your Arduino to send data from physical inputs (ie. variable resistors, buttons, tilt switch) to Processing. Receive the data and express it visually in your Processing sketch.

-Reflect on serial communication.

-❤️

 

Step1: Processing To arduino
  • Idea

Basically, I want to make a mini traffic light which can be manipulated by user. Given that I need to control three objects(LED) independently, I decide to use keyboard as a interactive tool. So my idea is that if user presses the button R, then the red LED will light and if presses other key(G or Y), then the according LED will light. And if no key is pressed, then all LEDs go out.

 

  • Physical Part

Due to my carelessness…this is the first time that I killed an innocent LED…SORRY!

 

  • Problem and Solution

My problem is simple–it just doesn’t work! Only the green light can work but very improperly. The rest of the LEDs just cannot work at all. However, after carefully debugging, the solution is also simple–delete a semicolon in one statement for a if condition 😂. Though this “bug” is rather small itself and is merely a consequence of my carelessness, in the process of finding this mistake, thanks to Nick, I have really achieved a lot! Most of them are some details and useful techniques:

  1. As is depicted in the image above, since what I want to “tell” my Arduino is a character (‘R’ or ‘G’ or ‘Y’), it is better to set up a character variable and let “myPort.write (that character variable)”. As is in my sketch, I use “number” as a character variable instead of using the lines in comment. And it is especially good for debugging with println! Though sometimes these two ways produce the same result, the new method suggested by Nick makes the sketch more organized and explicit.
  2. Print the variable you need in order to debug. When there is something wrong with my sketch, I am always so frustrated that I don’t know where to start. Now I know that printing the variable step by step is always a good way to debug.

Though these achievement may seem to be very “tiny”, I think they are very valuable for me because they are very helpful and can really help me to improve my ability to do coding! 😊

 

  • Final Outcome

This is how my project works and the complete codes are attached in the end.

 

Step2: ARDUINO to Processing
  • Idea

I have made a flower when we just began learning processing:

From then, I have always been thinking that can I do more to this flower? Can I change its color?Can I change the size of the petals? Can I make a kaleidoscope? And I am very happy that today is the day for me to realize all my dreams! So my idea is to make a simple kaleidoscope with potentiometer!

 

  • Physical Part

I know it is a little bit too simple…😂

But but but! This potentiometer has three different functions in different conditions and you can see it in the final outcome part! Little body with limitless power!

 

  • The Process

I made this project all by myself and I really enjoy the process of exploring for that you never know what new ideas will occur to you while exploring.

Here is how the project works and I encounter almost all these ideas while exploring.

 

1. Press the key ‘a’ and switch the potentiometer: The initial window has nothing but a yellow circle in the center, but when you press the key ‘a’ and switch the potentiometer, some ellipses will gradually emerge and I made this by means of setting a variable for the opacity.

 

2.  Press the key ‘s’ and switch the potentiometer: The size of the petals will change and it looks like a simple  kaleidoscope!

 

 

3.  Press the key ‘r’ and switch the potentiometer: This time, the color of the flower will change randomly and the value of the potentiometer will change the size of the flower. It is like a combination of all the functions above and I really like it.

 

For all these three functions I have applied a similar idea to them that is to int five variables r,g,b,d,x and control them separately and in different ways. Give the value of the potentiometer to different variables to realize different functions.

<fill(r,g,b,x) and the height of the ellipse is d+10>

 

  • Final Outcome

I really love it and I am happy that I can gradually realize the dreams that I used to be unable to realize!!! 

 

reflection

 

  • What are the possibilities for interaction that you can envision? 

I think since iphoneX and many other smart phones have already applied the technology of recognizing people’s faces, I think it may be possible in the future to build interaction with people’s face expressions. For example, connect a small camera to Arduino and it can sense people’s expression and it will create something accordingly in Processing–a smile creates a flower, tears create rain and so on.

Another idea is that maybe through camera as well, the computer can sense people’s motion and appearance and produce “a colon man” on the screen like a mirror! I think it maybe very interesting but difficult as well.

 

  • What are some of the benefits of controlling physical elements with computational media and of controlling screen-based computational media through physical interaction? 

I think the biggest benefit is that it breaks the seemingly absolute limitation between physical world and the world in computational media. As we all know, computational devices are very accurate, thus, by using computational media to control physical objects, the physical objects can conduct some very accurate behaviors. Also, since writing and adjusting sketches on computational media is relatively easier than making real physical things again and again, so the combination of these two is better than one.

In regard to controlling screen-based computational media through physical interaction, it can provides users with a fantastic feeling of interaction because they are actually touching or pressing something. And the computational media can create something that is hard for physical tools to create such as drawing a large quantity of circles with same color, it has essentially expanded creators’ imagination and has amplified people’s experiences!

 

  • Can you think of real world applications that take advantage of communication between the physical and screen-based computational media?

Smart phone. People can touch the screen or press some buttons to control the phone and the screen of the phone will give responses accordingly.

Computer. People can use mouses and press keyboard to control the computer such as what I am doing now–typing on keyboard and the character will appear on the screen.

 

 

//STEP 1__From processing to Arduino__Arduino's sketch

char valueFromProcessing;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode(9, OUTPUT);
  pinMode(10, OUTPUT);
  pinMode(11, OUTPUT);
}

void loop() {
  // put your main code here, to run repeatedly:
  while (Serial.available()) {
    valueFromProcessing = Serial.read();
  }

  if (valueFromProcessing == 'R') {
    digitalWrite(9, HIGH);
    digitalWrite(10, LOW);
    digitalWrite(11, LOW);
  } else if (valueFromProcessing == 'Y') {
    digitalWrite(10, HIGH);
    digitalWrite(9, LOW);
    digitalWrite(11, LOW);
  } else if (valueFromProcessing == 'G') {
    digitalWrite(11, HIGH);
    digitalWrite(9, LOW);
    digitalWrite(10, LOW);
  } else if(valueFromProcessing=='L'){
    digitalWrite(9,LOW);
    digitalWrite(10,LOW);
    digitalWrite(11,LOW);
  }
  delay(10);
}

---------------------------------------------------------------------------------------
//STEP 1__From processing to Arduino__Processing's sketch

import processing.serial.*;

Serial myPort;
int valueFromArduino;
char number;

void setup() {
  myPort = new Serial(this, Serial.list()[3], 9600);
  number='L';
}


void draw() {
  if (keyPressed) {
    if (key=='R') {
      //myPort.write('R');
      number='R';
    } else if (key=='Y') {
      //myPort.write('Y');
      number='Y';
    } else if (key=='G'){
      //myPort.write('G');
      number='G';
    }else{
      number='L';
    }
  }      
  myPort.write(number);
  println(number);
}

---------------------------------------------------------------------------------------
//STEP 2__From Arduino to Processing__Arduino's sketch
int sensorValue =0;
int outputValue=0;


void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);

}

void loop() {
  // put your main code here, to run repeatedly:
  sensorValue = analogRead(A0);
  int outputValue=map(sensorValue,0,1023,0,255);
  Serial.write(outputValue);
  
}


---------------------------------------------------------------------------------------
//STEP 2__From Arduino to Processing__Processing's sketch
import processing.serial.*;

Serial myPort;
int valueFromArduino;
int r,g,b,x,d;

void setup(){
   
size (600,400);
background(#F0FFFF);
printArray(Serial.list());
myPort= new Serial(this, Serial.list()[3],9600);
}

void draw(){
 while (myPort.available()>0){
  valueFromArduino=myPort.read();
}
noStroke();
fill(#FFD700);
ellipse(width/2,height/2,60,60);



fill(r,g,b);
ellipse(400,200,100,d);
ellipse(200,200,100,d);
ellipse(300,103,d,100);
ellipse(300,300,d,100);

pushMatrix();
translate(225,119);
rotate(radians(55));
ellipse(10,0,100,d);
popMatrix();

pushMatrix();
translate(0,265);
rotate(radians(45));
ellipse(10,0,100,d);
popMatrix();

pushMatrix();
translate(365,137);
rotate(radians(305));
ellipse(10,0,100,d);
popMatrix();

pushMatrix();
ellipse(10,0,100,d);
translate(225,285);
rotate(radians(305));
popMatrix();

background(#F0FFFF);
noStroke();
fill(#FFD700);
ellipse(width/2,height/2,60,60);

fill(r,g,b,x);
ellipse(400,200,100,10+d);
ellipse(200,200,100,10+d);
ellipse(300,103,10+d,100);
ellipse(300,300,10+d,100);




pushMatrix();
translate(225,119);
rotate(radians(55));
ellipse(10,0,100,10+d);
popMatrix();

pushMatrix();
translate(359,268);
rotate(radians(45));
ellipse(10,0,100,10+d);
popMatrix();

pushMatrix();
translate(365,137);
rotate(radians(305));
ellipse(10,0,100,10+d);
popMatrix();

pushMatrix();
translate(225,285);
rotate(radians(305));
ellipse(10,0,100,10+d);
popMatrix();

if(keyPressed){
  if(key=='s'){
  d=valueFromArduino;
  }else if(key=='a'){
    r=240;
    g=140;
    b=195;
    x=valueFromArduino;
  }else if(key =='r'){
    r=int(random(0,255));
    g=int(random(0,255));
    b=int(random(0,255));
    x=80;
    d=valueFromArduino;
    
  }
  
}

}