Recitation 10: Making a Media Controller: Tianyu Zhang (Eric&Young)

The idea is from the flappy bird, and I decide to use the sound sensor as an input to control the movement of the pipixia. If the volume is high, then the speed of moving higher is fast. And only when there is a sound the pipixia can move forward.

The picture of pipixia is from the resource of class22.

(https://drive.google.com/file/d/117byvL82I-8BSEA8o7O5Cz3CviGxK5vF/view)

 

Video

(Fail)

(Success)

Problems:

The first problem I met is that I can not set the lowest boundary to certain sections, and when I set the boundary, the pipixia would jump from one step to another step.

 

solution:

I use “if” statement and use “b=1” to judge whether the pipixia is on the step or not.

 

 

//processing

import processing.serial.*;
PImage ppx;
int poi;
float a;
float h=500.0;
int b=0;

Serial myPort;
int valueFromArduino;


void setup() {
  size(1500, 550);
  background(255);
  printArray(Serial.list());
  ppx=loadImage("pipi.jpeg");
  printArray(Serial.list());
  // this prints out the list of all available serial ports on your computer.

  myPort = new Serial(this, Serial.list()[0], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.
}


void draw() {
  background(255);
  // to read the value from the Arduino
  while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
  }
  println(a);
  //println(valueFromArduino);//This prints out the values from Arduino
  a=map(valueFromArduino, 0, 1023, 0, 25);
  if (a>0.01) {
    poi+=1;
  }


  //gravity
  h++;
  h=h-a;
  if (poi>0&&poi<300) {
    if (h>=500) {
      h=500;
    }
  }
  //if (poi>250&&poi<350) {
    if (h==550) {
      background(255);
      text("YOU LOSE!", 550, 250);
    }
  //}  
if (poi ==400 && h<450) {
  b=1;
}
  if (poi>400&&poi<600&&b==1) {
    if (h>=450) {
      h=450;
    }else{
      h++;
    }
  }
if (poi ==650 && h<250) {
  b=1;
}
  if (poi>650&&poi<800&&b==1) {
    if (h>=250) {
      h=250;
    }
  }
  if (poi>800&&poi<1100) {
    if (h>=500) {
      h=500;
    }
  }
  if(poi==1100&&h<400){
  if (poi>1100&&poi<1300) {
    if (h>=400) {
      h=400;
    }
  }}else{
  h++;}
  if (poi>1300&&poi<1500) {
    if (h>=250) {
      h=250;
    }
  }
  if (poi>=1500) {
    poi=1500;
    text("YOU WIN!", 550, 250);
  }
  if (h>550){
      background(255);
      text("YOU LOSE!", 550, 250);
  }
  println(h);
  noStroke();
  fill(0);
  rect(0, 500, 300, 50);
  rect(400, 450, 200, 50);
  rect(650, 300, 150, 50);
  rect(800, 500, 300, 50);
  rect(1100, 400, 200, 50);
  rect(1300, 250, 200, 50);
  image(ppx, poi, h-50, 50, 50);
}  



//Arduino

const int pinAdc = A0;
void setup() {
  Serial.begin(9600);
}


void loop() {
  long sum = 0;
    for(int i=0; i<32; i++)
    {
        sum += analogRead(pinAdc);
    }

    sum >>= 5;

    Serial.println(sum);
  int sensorValue = analogRead(A0) / 4;
  Serial.write(sum);

 delay(10);
}

Recitation 10: Making a Media Controller–Daisy Chen(Marcela)

Recitation Excercise:

My media controller is made of the potentiometer, which can send values from Arduino to Processing and control certain characteristics of the video, for example, the speed. Meanwhile, it can jump to certain positions of the video and add tint effect on the video based on the value it sends.

 

 

In processing, I set three conditions for the control of potentiometer. Since the value it sends range from 0 to 255, I set three conditions: VA<100, 100<VA<200, VA>200. I make the background color changes to different colors according to the value from Arduino. The speed is also controlled by this value. To illustrate this effect, I chose a video of a timelapse of a water drop. So by controlling the potentiometer, I can control the speed of the dropping process. Besides, the controller can make the video jump to a certain position according to the value. To avoid chaos, I make it only to run this effect when VA>150, so that the video won’t seem to play too randomly.

Video creadit: Christoph Pantel-drop

Reflction:

reading: Levin, G. Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers

This article of computer vision is really inspiring and insightful for interactive media students. It not only offers several examples to illustrate how technology can be used but also introduces how to problems that vision algorithms have been developed to address, and their basic mechanisms of operation.

My work in the recitation only touches the bottom line of this field of computer vision by adopting the basic idea of using the input (information) from a physical device to control something digital. Since this controller I made during the recitation directly send values to processing, there is nothing to do with the part of detecting motion. And among the examples the author gave in the article, I think, it’s the Kinect technology that brings those work to a higher interaction level. One project that impresses me a lot is LimboTime. In this project, two players stand at opposite sides of the field of vision of a regular webcam, with their fingers or arms extended towards each other. The game system locates their extended fingers and connects a horizontal line between them. This line continually connects these points, even if the players move their hands. A third player then tries to “limbo” underneath the imaginary line created by the first two players. This project is very interactive from my perspective since the whole game is “controlled” by the players. They build the “barriers” and others need to try to cross it, which makes the game very engaging and interesting.

The computer vision, as is shown in this article, is the bridge that connects the physical world and the digital world. Together with the workshop held by Bryan Chung which offered me lots of interesting insights, I’m looking forward to exploring more about this field in the future.

//Arduino
void setup() {
  Serial.begin(9600);
}
void loop() {

  int sensorValue = analogRead(A0) / 4;
  Serial.write(sensorValue);
  delay(10);
}
//processing
import processing.video.*;
Movie myMovie;

// This code receives one value from Arduino to Processing 

import processing.serial.*;

Serial myPort;
int VA;

void setup() {
  size(600, 480);
  myMovie = new Movie(this, "Christoph Pantel-drop.mp4");
  myMovie.loop();
  //
  printArray(Serial.list());
  // this prints out the list of all available serial ports on your computer.

  myPort = new Serial(this, Serial.list()[ 3 ], 9600);
}
void movieEvent(Movie movie) {
  myMovie.read();
}
void draw() {    
  image(myMovie, 0, 0, width, height);   
  float newSpeed = map(VA, 0, 255, 0.1, 5);
  myMovie.speed(newSpeed);

  if (VA<100) {
    tint(255, VA, 0);
  }
  if (VA>100||VA<200) {
    tint(VA, 150, VA);
  }
  if(VA>200){
    tint(0,VA,255);
  }

  while ( myPort.available() > 0) {
    VA = myPort.read();
  }
  println(VA);//This prints out the values from Arduino

  if (VA>150) {
    float duration=map(VA,0,255,25,0);
    myMovie.jump(duration);
  }
} 

Recitation10 (Leon)

In today’s class, I use potentiometer on Aduino side to send one value to Processing side, and use the value to control the size of the rectsize. I use the Realtime Pixels Manipulation on the Processing, so when you control the potentiometer, the rectsize will change, and thus the picture will be clear or blurry.

As what we read this week, “Computer vision algorithms are increasingly used in interactive and other computer-based artworks to track people’s activities. ” We can use the interaction between people and the two app to detect motion orpresence.

import processing.serial.*;
import processing.video.*;
Capture cam;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 1;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/


void setup() {
  size(640, 480); 
  cam = new Capture(this, 640, 480);
  cam.start();

  setupSerial();
}


void draw() {
  updateSerial();
  printArray(sensorValues);
if ( cam.available() ) {
    cam.read();
    cam.loadPixels();
  }
  int rectSize=sensorValues[0]/100;
  int w = cam.width;
  int h = cam.height;
  for (int y = 0; y < h; y+=rectSize) {
    for (int x = 0; x < w; x+=rectSize) {
      int i =  x + y * w;
      fill( cam.pixels[i] );
      rect(x, y, rectSize, rectSize);
    }
  }
  cam.updatePixels();
}

  // use the values like this!
  // sensorValues[0] 

  // add your code

  //




void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[0], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = 'n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Recitation 10: Making a Media Controller: Stanley (Cossovich)

For my Media Controller, I utilized a slide potentiometer to control the color filter overlaying my laptops face camera. While using potentiometers in previous recitations and projects, I always wanted to explore different types and uses of potentiometers, asides from the small ones included in the Arduino kit. When I discovered that the resource center had slide potentiometers, I wanted to integrate this into this project to explore a different type of controller interaction. I found the slide potentiometer to work more smoothly and accurately compared to the small default potentiometers. Additionally, it seemed simpler to interact with when considering ergonomics.

 

After reading Computer Vision for Artists and Designers,  I found inspiration from Myron Krueger’s Videoplace Project (1969-1975). His project consisted of a live camera capturing objects moving in its frame and then rendering them into the digital world while assigning solid colors for the different objects rendered. This is similar to my device in the sense that the software is utilizing colors in order to alter what is projected from the camera. However, his project is much more complex because it is not projecting the actual camera footage, but rather an animated one.

Arduino Code:

Processing Code (below):

import processing.video.*;
import processing.serial.*;
Capture cam;

Serial myPort;
int valueFromArduino;


void setup() {
  size(640, 480);
  cam = new Capture(this, 640, 480);
  cam.start();

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 9 ], 9600);
}


void draw() {
  background(0);
  if (cam.available()){
      cam.read();
  }
    tint(valueFromArduino, 200, 200);
    //tint(100, valueFromArduino, 100);
   // tint(200, 200, valueFromArduino);
    
    image(cam, 0, 0);
 
  while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
  }
  println(valueFromArduino);
}

Week 14, Recitation 10 (Leon)

Arduino: For the physical controller, incorporate digital and/or analog input to your Arduino. These inputs should have a clear correlation to however they manipulate the media in your Processing sketch. To make your controller, you may use components from your kit, or check out and use sensors from the equipment room.

【Since my breadboard was used to make the final project, I borrowed half of my friend’s:)】

Processing: You may choose to use video, live video via webcam, or images as the media you will control in Processing,. If you do not have authorship over the media that you use for this exercise, make sure you credit the source.

【Arduino Prat】

void setup() {
  // put your setup code here, to run once:
Serial.begin(9600);
}

void loop() {
  // put your main code here, to run repeatedly:
  int sensor1 = analogRead(A0);
  int sensor2 = digitalRead(3);
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.println();
  delay(100);
}


【Processing Part】

import processing.video.*; 
import processing.serial.*;

String myString = null;
Serial myPort;

int r, g, b;
int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/


Capture cam;
void setup() { 
  size(640, 480); 
  cam = new Capture(this, 640, 480);
  cam.start();
  setupSerial();
} 
void draw() {
  updateSerial();
  printArray(sensorValues);
  if ( cam.available() ) {
    cam.read();
    cam.loadPixels();
  }
  int rectSize = 15;
  int w = cam.width;
  int h = cam.height;
  for (int y = 0; y < h; y+=rectSize) {
    for (int x = 0; x < w; x+=rectSize) {
      int i =  w - x - 1 + y * w;
        if(sensorValues[1]==0){
         r = int( red(cam.pixels[i])); 
         g = int( green(cam.pixels[i]));
         b = int( blue(cam.pixels[i]));}
                if(sensorValues[1]==1){
         g = int( red(cam.pixels[i])); 
         b = int( green(cam.pixels[i]));
         r = int( blue(cam.pixels[i]));}
        if (x < width*1/3) {
          cam.pixels[i] = color(r, g, +sensorValues[0]/10);
        } else if (x < width*2/3) {
          cam.pixels[i] = color(+sensorValues[0]/10, g, b);
        } else {
          cam.pixels[i] = color(r, +sensorValues[0]/10, b);
        }
      fill( cam.pixels[i] );
      rect(x, y, rectSize, rectSize);
    }
  }
  cam.updatePixels();
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 25 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = 'n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Recitation 10 Documentation – Ke Xu (Leon)

Recitation Exercise

My Project:

I use the potentiometer to control the clarity of the image on the Processing sketch. The image I chose is a picture of two polar bears. My inspiration is not only the fact that it is my favorite animal but also the temperature today. Although today is the end of November, the temperature is still warm which may be influenced by global warming and the emission of greenhouse gases. The floating ice in the Arctic pole is melting and the polar bears are faced with threat of extinction. So our control of potentiometer is like human beings’ behavior in daily life. We can save these lovely animals or damage them. If we take actions from now on, we may keep them as lively as depicted in the picture.

Arduino:

Components:

1* Arduino uno

1* potentiometer

I used the potentiometer to produce different values to be transported to processing to control the clarity of the polar bear image. I used the sample one value serial communication example to help me finish coding. But I use Serial.write code because in that way the processing can read the value efficiently.

Processing:

The picture I used: https://commons.wikimedia.org/wiki/Ursus_maritimus#/media/File:Ursus_maritimus_us_fish.jpg

I think the processing is much more complicated than Arduino part. The picture I chose is my favorite animal. When I rotate the potentiometer, I can see the picture clearer. At first I didn’t realize the value received from Arduino ranged from 0 to 255, so the ellipse is always too large. Then I used the Map function to adjust the size of ellipse. However, I think the potentiometer is not the best choice for this exercise, because the accuracy is not high enough and sometimes the processing stretch does not work as expected. Maybe to use Arduino just for turning on and off and control the change of value in the processing code will increase the accuracy.

 

A reflection about the ways technology was used in my final project:

As for the final project, which is our car with searching, catching and lifting module, the main technology we use is the ir distance sensor. The interaction between user and the computer and car is presented through user’s control of car in order to find the dropped object and get it up with the only help of the direction presented on the computer screen. At first, we believe it is pretty cool because the interface we design is really fancy and at that time we use the ultrasonic distance sensor because we know how to use it through online sources. However, as this weeks reading has mentioned:

“Unlike the human eye and brain, no computer vision algorithm is completely “general”, which is to say, able to perform its intended function given any possible video input. Instead, each software tracking or detection algorithm is critically dependent on certain unique assumptions about the real-world video scene it is expected to analyze. If any of these expectations is not met, then the algorithm can produce poor or ambiguous results, or even fail altogether. For this reason, it is essential to design physical conditions in tandem with the development of computer vision code, and/or to select software techniques which are best compatible with the available physical conditions.”

We finally find our physical condition is not appropriate for the software or the code we design because with the rotation of the sensor, it is very likely for the car to miss the object. Even though the interface is really cool, it does not fulfill our need. So we gave up that idea and chose to use five ir distance sensors and another version of code to achieve synchronous result of sensor on the screen. I think this fully shows how important it is to choose software techniques which are best compatible with the available physical conditions.

 

//For Arduino

void setup() { 
   Serial.begin(9600); 
} 

void loop() { 
   int sensorValue = analogRead(A0);      
   Serial.write(sensorValue);
   delay(1); 
}

//For Processing

PImage img;
import processing.serial.*;


Serial myPort;
int valueFromArduino;

void setup() {
  size(977,639);
  noStroke();
  img = loadImage("bear.jpg");
   printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[17], 9600);
}

void draw() {
  for (int i=0; i<100; i++) {
    int size = valueFromArduino;
    map(size,0,255,6,40);
    int x = int( random(img.width) );
    int y = int( random(img.height) );
    color c = img.get(x, y);
    fill(c);
    ellipse(x, y, size, size);}
    
    while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
  }
  //println(valueFromArduino);
  
 }


void mousePressed() {
  saveFrame("mySketch.png");
}

#Recitation9-Winny Wang

Recitation 9: Serial Communication

Date: 30 Nov 2018

Documented by: Winny Wang

Instructor: Marcela Godoy

Exercise 1: Make a Processing Etch a Sketch

For this exercise, using Arduino to send two analog values to Processing via serial communication wasn’t very hard for me. I finished my code and built the circuit pretty quick but then I didn’t get the idea how “drawing” work, so I ended up like this:

After a peer corrected me, it can successfully started to draw. But another problem hit me that somehow the line drawn by the movement of the ellipse is not continuous, it is still like many dots placed together, shown as the pic here:

So I asked for help from fellow and learned a new function called “easing”, which I can set the locations of a starting point and an ending point, the object would move towards its destination at a faster speed, shown as the code down there. The problem is caused by the speed of my laptop which is not as fast as the speed of drawing. Using “easing” to boost the speed makes it working.

 

 

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;  /** this array stores values from Arduino **/
float x;
float y;
float easing = 0.05;

void setup() {
  size(500, 500);
  background(255);
  setupSerial();
  frameRate(120);
}


void draw() {

  updateSerial();
  //printArray(sensorValues);

  // use the values like this!
  // sensorValues[0] 
  float tx = map(sensorValues[0], 0, 1023, 0, width);
  float ty = map(sensorValues[1], 0, 1023, 0, height);
  println("x: " + tx + "y:" + ty);

  float targetX = tx;
  float dx = targetX - x;
  x += dx * easing;

  float targetY = ty;
  float dy = targetY - y;
  y += dy * easing;
  fill(0);
  ellipse(x, y, 5, 5);


  // add your code

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[4], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = 'n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Recitation ten documentation (Qiwei Chen)

All materials and components used in this recitation:

  • 1 * Arduino
  • 1 * Breadboard
  • 2 * Potentiometer
  • 1 * Laptop
  • Jumper Cables

Exercise:

I made a device that incorporates both Arduino and Processing that when I turn the potentiometers connected to Arduino, the image shown on screen by Processing would change color.  Video:



import processing.serial.*;


String myString = null;
Serial myPort;

PImage img;
int NUM_OF_VALUES = 2;  
int[] sensorValues;  

void setup() {
  size(1500, 300);
  background(0);
  setupSerial();
  img = loadImage("elementsMobile.png");
  img.resize(1500,300);
}


void draw() {
  updateSerial();
  printArray(sensorValues);
  tint(sensorValues[0]/4,sensorValues[1]/4,0);
  image(img,0,0);
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[0], 9600);

  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = 'n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

week 9

First time I saw the finished project on the instruction website, I thought it was so hard but when I browsed the example coding, I tried to recall the coding I learnt in lectures. I just used the mouse pressed and mouse moving coding to connect processing with Arduino.

The second project was so amazing when I was moving the mouse and the colors were changing.

IMG_7474

IMG_7475

*The files are so large so they cannot be viewed in this window.*