Final Project (Rudi); DIY Disco

  1. inspiration

We were inspired by both dance revolution and a launchpad. Both of them are regarding one kind of creativity: making a dance and making music. We wanted something that can have both, but without professional knowledge about dance, music or any equipment.

We wanted to connect more to our generation by using the form of dance revolution, and also wanted to make it personal. So we thought of a way to make it different for every user.

So we thought rather than stepping on the panels as told by the screen, the user initiating a sound and dance and resulting in their own music and choreography would be a nice twist.

  1. call for sound file (from 99 sounds)

I tried to put the sound file inside the class I made,

popUp(String imageName, String soundName) {
img = loadImage(imageName);
s = new SoundFile(this, soundName);
}

but when the library call for “this“, it called for not the sound file but the class(popUp),

so with the help of Jiwon, I pulled it outside the class to the main Tab. I made a array of sound files and called for them outside the class first, then call the elements of the array into the class.

SoundFile [] sounds = new SoundFile[9];
sounds[0] = new SoundFile(this, “backgroundMusic.wav”);
Pop = new popUp(“happyCat.jpg”, sounds[0]);

2. add a vibration sensor

This part was same as the recitation 10, so I’ll link it to the recitation documentation.

Basically,  we detected movement on the vibration sensor and sent it to processing. In processing, the image and the sound pops up only when the vibration is received from arduino.

http://ima.nyu.sh/documentation/2018/04/29/lab-11-media-controller-rudi/ ‎

We added 9 more sounds and 9 more sensors to the recitation code.
Because we were using the class 9 times, we changed

Pop = new popUp(“man.jpg.png”, sounds[0]);

to

Pop0 = new popUp(“man.jpg.png”, sounds[0]);
Pop1 = new popUp(“man2.jpg.png”, sounds[1]);
Pop2 = new popUp(“man3.jpg.png”, sounds[2]);
….(continues)

So that processing can operate different image and sound for different sensors.

3. sending the sensors correctly

When we tried to add a sensor, we realized that we are not sending the right information.
We were sending:

Serial.println(analogRead(A0), DEC);
if (analogRead(A0) > 1) {
Serial.println(1);
}
Serial.println(analogRead(A1),DEC);
if (analogRead(A1) > 1) {
Serial.println(2);
}

which we changed with the help of Nick to:

int sensor1 = analogRead(A0)/4;
int sensor2 = analogRead(A1)/4;

Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.println();

for the Processing to receive it as an array written as the IMA Serial Communication example.

4.  Calling for video and sound file

Then we tried to change the class from calling one image to calling two, and flipping the two images like an gif. However, in the process, calling for two images and flipping them did not work well. So we changed to calling for a video (gif converted to video) and a sound file, and got rid of the PopUp class.

After getting one sensor to work, it was fairly easy to do eight more.

5. Change to FSR sensor

Because we planned as the users standing on the first sensor while pressing other sensors with one foot, Nick advised that using FSR sensor would be more suitable than a vibration sensor.

6. Physical component

We used two large cardboards, one as a background board (colored black) and one to cut out the panels. We painted them with spray paint and drew footprint to indicate where to step on.

 

 

 

The vibration sensors are attached under the panel and connected to the breadboard and the arduino mega with long cords. We found out that if the FSR is kept pressed on, the music starts over and over. So we left the center panel sensor-less, so that the user could stand on without triggering any sound or video. We first thought to make a start panel outside the 3X3 panels, but realized it would confuse the users when to press it and what function it has. It also might result in the user accidentally pressing it more than once and the background music not properly playing. So we removed the first sensor completely from the board, and we pressed it instead to monitor and control user experience.

7. User testing

Here is the demo of the prototype version:

Here is the set up on the IMA show:

Here are the videos of users with our project:

IMG_8844-2

IMG_8854

IMG_8861

IMG_8860

IMG_8862

IMG_8867-2

The unexpected points from the user testing was:

  • Users thought they should follow the dance moves on screen
  • Users did not pay much attention to the background music or how well the sound effects suited the music
  • Users stepped on the panels VERY FAST and VERY HARD (even after our demo) and wanted to step on more than one at the same time

Our improvements could be:

  • We add another function to control the light –complete the “dance floor” ambience
  • Capture the user’s face and paste their face on to the dancing character
  • Remember each user’s moves and play it on the side of the screen — all together: a dance floor!
  • enhance the feeling of composing music so that users do not only focus on the dance moves
  • use stronger sensors that is okay to be stomped on
  • Divide arms/ body/ legs to combine different dance moves at the same time
---------------------Arduino--------------------------
int val1, val2, val3, val4, val5, val6, val7, val8, val9 = 0;
int a1, a2, a3, a4, a5, a6, a7, a8, a9 ;


void setup() {
  Serial.begin(9600);
}

void loop() {

  a1 = analogRead(A0);
  a2 = analogRead(A2);
  a3 = analogRead(A3);
  a4 = analogRead(A4);
  a5 = analogRead(A5);
  a6 = analogRead(A6);
  a7 = analogRead(A7);
  a8 = analogRead(A8);
  a9 = analogRead(A9);

  if (a1 > 300) { //first sensor
    val1 = 1;
  }
  else if (a1 <= 300) {
    val1 = 0;
  }

  if (a2 > 30) { //second sensor
    val2 = 1;
  }
  else if (a2 <= 30)  {
    val2 = 0;
  }

  if (a3 > 20) { //third sensor
    val3 = 1;
  }
  else if (a3 <= 20)  {
    val3 = 0;
  }

  if (a4 > 20) { //fourth sensor
    val4 = 1;
  }
  else if (a4 <= 20)  {
    val4 = 0;
  }

   if (a5 > 20) { //fourth sensor
    val5 = 1;
  }
  else if (a5 <= 20)  {
    val5 = 0;
  }

   if (a6 > 20) { //fourth sensor
    val6 = 1;
  }
  else if (a6 <= 20)  {
    val6 = 0;
  }
  
   if (a7 > 20) { //fourth sensor
    val7 = 1;
  }
  else if (a7 <= 20)  {
    val7 = 0;
  }

 if (a8 > 20) { //fourth sensor
    val8 = 1;
  }
  else if (a8 <= 20)  {
    val8 = 0;
  }

   if (a9 > 20) { //fourth sensor
    val9 = 1;
  }
  else if (a9 <= 20)  {
    val9 = 0;
  }

  Serial.print(val1);
  Serial.print(",");
  Serial.print(val2);
  Serial.print(",");
  Serial.print(val3);
  Serial.print(",");
  Serial.print(val4);
  Serial.print(",");
  Serial.print(val5);
  Serial.print(",");
  Serial.print(val6);
  Serial.print(",");
  Serial.print(val7);
  Serial.print(",");
  Serial.print(val8);
  Serial.print(",");
  Serial.print(val9);
  Serial.println(); //add line to end the series of values

  delay(100);

}



------------------------------------Processing------------------------------

import processing.sound.*; //calling sound library
import ddf.minim.*; //calling minim(another type of sound) library
import processing.video.*; //calling video library
import processing.serial.*; //connecting to arduino for serial communication


Minim minim; //basics for minim
AudioPlayer groove;


String myString = null;
Serial myPort; 
//int oldValue=0; //value updated by the sensorvalues received from arduino
boolean play1, play2, play3, play4, play5, play6, play7, play8 = false; //boolean to turn on and off the video

int NUM_OF_VALUES = 9;   //number of values received
int[] sensorValues;      //this array stores values from Arduino
SoundFile [] sounds = new SoundFile[8]; //array of soundfiles
Movie [] danceMoves = new Movie[9]; //array of dancemoves videos



void setup() {
  setupSerial();  //set the basics
  size(700, 800);
  background(255);

  //videos array
  danceMoves[0] = new Movie(this, "basicPose.mp4");
  danceMoves[1] = new Movie(this, "2whips.mp4");
  danceMoves[2] = new Movie(this, "dab.mp4");
  danceMoves[3] = new Movie(this, "floss.mp4");
  danceMoves[4] = new Movie(this, "twerk.mp4");
  danceMoves[5] = new Movie(this, "naenae.mp4");
  danceMoves[6] = new Movie(this, "funkychickendone.mp4");
  danceMoves[7] = new Movie(this, "blocboy.mp4");
  danceMoves[8] = new Movie(this, "gangmanstyle.mp4");
  

  minim = new Minim(this); //using the minim library
  groove = minim.loadFile("shortBackgroundMusic.mp3", 2048); //calling the background file

  //sounds array
  sounds[0] = new SoundFile(this, "bell.wav");
  sounds[1] = new SoundFile(this, "fireCrack.wav");
  sounds[2] = new SoundFile(this, "Boing.wav");
  sounds[3] = new SoundFile(this, "Drum.wav");
  sounds[4] = new SoundFile(this, "Error02.wav");
  sounds[5] = new SoundFile(this, "cymbals.wav");
  sounds[6] = new SoundFile(this, "computerError.wav");
  sounds[7] = new SoundFile(this, "zzHat.wav");
}



void movieEvent(Movie m) { //using the video library
  m.read();
}



void draw() {
  updateSerial();
  //background(255);
  printArray(sensorValues); //print the values received from arduino


  //first sensor
  if (sensorValues[0]==1) {  
    danceMoves[0].loop(); //calling for the video
    image(danceMoves[0], 0, 0, 700, 800); 
    groove.loop(10); //the sound on loop
  }

  //second sensor
  float mt1 = danceMoves[1].time(); //get the playing time of the video
  //if (sensorValues[1]!=oldValue) { //conditions for the first sensor    
  if (sensorValues[1]==1) { //receive value
    play1 = true;
    sounds[0].play();
  }
  //  oldValue=sensorValues[1];
  //}
  if (play1 == true) { //play the video
    danceMoves[1].play();
    image(danceMoves[1], 0, 0, 700, 800);
    if (mt1 >= 0.8) { //stop the video if the playing time is longer than the duration of the video
      //danceMoves[1].jump(0);
      danceMoves[1].stop(); 
      play1 = false; //stop the video
    }
  }

  //the third sensor
  float mt2 = danceMoves[2].time(); 
  if (sensorValues[2]==1) { //receive value
    play2 = true;
    sounds[1].play();
  }
  if (play2 == true) { //play the video
    danceMoves[2].loop();
    image(danceMoves[2], 0, 0, 700, 800);
    if (mt2 >= 0.467) { //stop the video
      danceMoves[2].stop();
      play2 = false;
    }
  }

  //the fourth sensor
  float mt3 = danceMoves[3].time(); 
  if (sensorValues[3]==1) { //receive value
    play3 = true;
    sounds[2].play();
  }
  if (play3 == true) { //play the video
    danceMoves[3].loop();
    image(danceMoves[3], 0, 0, 700, 800);
    if (mt3 >= 0.467) { //stop the video
      danceMoves[3].stop();
      play3 = false;
    }
  }
  
  //the fifth sensor
  float mt4 = danceMoves[4].time(); 
  if (sensorValues[4]==1) { //receive value
    play4 = true;
    sounds[3].play();
  }
  if (play4 == true) { //play the video
    danceMoves[4].loop();
    image(danceMoves[4], 0, 0, 700, 800);
    if (mt4 >= 0.467) { //stop the video
      danceMoves[4].stop();
      play4 = false;
    }
  }
  
  //the sixth sensor
  float mt5 = danceMoves[5].time(); 
  if (sensorValues[5]==1) { //receive value
    play5 = true;
    sounds[4].play();
  }
  if (play5 == true) { //play the video
    danceMoves[5].loop();
    image(danceMoves[5], 0, 0, 700, 800);
    if (mt5 >= 0.467) { //stop the video
      danceMoves[5].stop();
      play5 = false;
    }
  }
  
  //the seventh sensor
  float mt6 = danceMoves[6].time(); 
  if (sensorValues[6]==1) { //receive value
    play6 = true;
    sounds[5].play();
  }
  if (play6 == true) { //play the video
    danceMoves[6].loop();
    image(danceMoves[6], 0, 0, 700, 800);
    if (mt6 >= 0.467) { //stop the video
      danceMoves[6].stop();
      play6 = false;
    }
  }
  
  //the eighth sensor
  float mt7 = danceMoves[7].time(); 
  if (sensorValues[7]==1) { //receive value
    play7 = true;
    sounds[6].play();
  }
  if (play7 == true) { //play the video
    danceMoves[7].loop();
    image(danceMoves[7], 0, 0, 700, 800);
    if (mt7 >= 0.467) { //stop the video
      danceMoves[7].stop();
      play7 = false;
    }
  }
  
  //the ninth sensor
  float mt8 = danceMoves[8].time(); 
  if (sensorValues[8]==1) { //receive value
    play8 = true;
    sounds[7].play();
  }
  if (play8 == true) { //play the video
    danceMoves[8].loop();
    image(danceMoves[8], 0, 0, 700, 800);
    if (mt8 >= 0.467) { //stop the video
      danceMoves[8].stop();
      play8 = false;
    }
  }
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = 'n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = 'n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Leave a Reply