[NIME]Final Instrument(Cyndi)

Date: Dec.14th

Name: Cyndi

Instructor: Antonius

Goal: To create an instrument that accompanies the melody

Material used: wires, makey makey, electrical tape, cardboard, conductive fabric, normal tape, drilling machine, midi keyboard


At the very beginning, I wanted to create an instrument that imitates the piano. I wanted to use the makey makey to imitate the right hand on the piano, which plays the main melody and leap motion also the accelerometer for the left hand to play the accompaniment. After talking to professor Antonius, I decided to borrow a midi keyboard from ima equipment room and only use makey makey for my project, since simply making a keyboard by myself isn’t a new interface, and time was really limited. The midi keyboard is just super cool!!!!

So I started to think about what is really cool and new for an instrument. When I was doing the final concept presentation, a professor from Abu Dhabi actually inspired me a lot. He suggested me that I could use my body to interact with the instrument since the playing of piano consists of both hand and feet. I decided to add the feet part in my instrument, so I created four pedals, which represent four pieces of accompaniment. I took cardboard from the box room and cut it into four pieces. I wanted to stick them to the ground, and every time I step on them, different accompaniments will be triggered. To make this happen, at the beginning, I connected them to makey makey using the female-male wires, but it turned out to be really loose. Then I thought about soldering them together, but it still fell off easily. Professor Antonius suggested me to plug makey makey into a mini breadboard, which is very convenient and saved a lot of time for me.For the second part of my instrument, I wanted to interact with the audience. I will grab a wire that is connected to the ground, and they will grab a wire connected to specific keys on makey makey, so when I touch him or her, we are connected and different chords will be played.

After all the things were prepared, I moved to 825 and started working on it. I taped the pedals on the ground and connected each of them with two wires-one to the ground and one to the key on makey makey. I used super long wires for each of them because makey makey has to be connected to the computer. After connected all of them together, I had 8 messy wires crossing each other and it looked very ugly. I still have five more wires to go because I still need to do the touching part. I used a whole afternoon to strip and plug all the wires. However, when it came to the rehearsal, there was a huge problem with my instrument. Since all the wires are crossing each other and I almost over stripped most of the wires, every wire was touching each other and extra sounds were triggered all the time, which really influenced my performance. After talking to my friends, I decided to wrap electrical tape around all the connected part to try to avoid the extra sounds.

It worked well that night. However, when I came back on Friday morning, everything was wrong again. I was nearly mentally breakdown at that time because it was so close to the final performance. After talking to Antonius, he suggested me to twist the wires together and only leave a short part for them to connect to the breadboard. It worked amazingly good and nearly no extra sounds were made during the pedal part of my instrument. I wanted to tape the conductive fabrics on my shoes at first, but then I found out it was too obvious and also ugly, so I decided to take Antonius’s advice and use my bare foot. However, when I tried the pedals with my bare foot, it seemed like sometimes the connection was not sensitive enough. With the help of professor Antonius, I put some conductive fabric under the wires, and the connection suddenly became really strong.


Then I started to make the chords part. I connected the wires to four arrows on makey makey and one wire to the ground. I actually wanted my friends to sit on the table and hold a shiny star, but there were metal parts on the table which can be easily connected to my friends so extra sounds are being made again and again. I had to give up this idea. Instead of letting them sit on the table, I decided to let them stand on the ground and each of them will hold a star for me. For the star part, originally I wanted them to be shiny, so I talked to Antonius and borrowed led lights, but it didn’t work well because the effect is not obvious. At last, I decided to print out some yellow stars and put them on the wires as a decoration.

However, extra sounds after playing again when I added the chords part. To reduce the extra sounds, professor Antonius helped me put a switch in my code. Only when I press k on the keyboard will the chords start to play. It worked really well. Also, to make the wires organized and protect makey makey, Antonius helped me to tape the makey makey right next to my computer.

For the coding part, I used minim library to play all the sounds I pre-recorded. One thing about minim library is that the play function only play the music once, so if I want the music to play again and again with my movement, I had to put a rewind function.

One really tricky problem in the process is that when I press the key for a long time and then press it again, the music will not continue to play. At first, I thought there was something to do with makey makey, but the keys on the computer didn’t work either. With the help of professor Antonius, I found out that my keyboard stopped typing when I press one certain key for a long time and then press it again. The problem is that I updated to mac sierra and the keyboard setting actually changed when I press a key for a long time. Although this has nothing to do with coding, this is still a great lesson and taught me that I should never let go of any detail in the process of making my project.

Finally, my instrument was finished. The performance went way better than I imagined. Everything just worked really well!!!!! I was so thrilled about the fact that I did all of this!!!


I really learned a lot from this process. I am much more familiar the arrangement of the wires and also the coding, and I broke my limit and achieved to extend my ideas to something cool, and I actually did it! Every time when I thought my instrument was finished, new problems popped up. I have to admit that I was mental breakdown for a plenty of times, but I overcame all of them and was satisfied with what I made. I am just so happy for all of us. The performance was amazing! I also learned that the final project is always not what you want to make at the very beginning. It always change along with the process. Instead of keep thinking about the original idea, embrace changes and more possibilities may be a better way to deal with it. Last but not least, I just want to thank everyone, especially professor Antonius, for helping me out during the process. I cannot make it only by myself.

[NIME] Final Instrument: The Umbrello

My final instrument for NIME is called “the Umbrello” — which is a play on the words “umbrella” and “cello”.

The idea came to me in two parts: first, I wanted to build upon the interaction mode I used in my Musical Wands; second, I had an umbrella I did not need lying around, and thought it would make an interesting instrument. After playing around with the umbrella and wands for a while, I came up with a general idea of how I could turn them into an instrument, and presented that in class. The way I first envisioned it was by holding the umbrella open and then playing it using a chopstick-made wand.

However, based on the feedback I received in class and the brainstorming process that followed, I fleshed out my idea some more and decided to instead keep the umbrella closed as my default, and create an actual “bow”, so that the instrument is more similar to a real violin/cello.

I started the work with the umbrella. The plan was to add a different amount of resistance to each wire, and then connect all of them together to the “ground” (0V). This meant twisting a lot of resistors together, and great amounts of conductive tape, conductive fabric, insulating tape, and… patience.

In the end, I succeeded in making the umbrella into the main part of my instrument. I then moved on to researching violin bows, and tried to recreate a simplified one in Illustrator so I can laser cut it. My final result was not perfect, but it looked pretty realistic, so it seemed promising.

As far as the code is involved, I decided to produce the sounds through Max MSP based on the inputs read from Arduino. Thus, the umbrella and bow are connected to the Arduino, which calculates the amount of resistance, identifies the wire being touched, and then sends that information to Max via serial communication. The Max part is rather convoluted, as I settled upon having 8 different chords which can be played (each umbrella wire corresponding to a different chord). I created each chord manually, using sliders and the “noteout” object, and although tedious, the result seemed rather good.

Based on the Professor’s feedback, I also decided to add some rain in my music. This proved much more difficult, but after a lot of “googling” I found a similar experiment on the internet, from which I learned the basics of making a rain synthesizer and then used a simplified version in my project. The rain is supposed to act primarily as background noise, but it is also influenced by the notes played, for improved cohesion with the overall melody.


NIME Final Instrument(Ronan)


For my final project, I wanted to make an instrument which can generate different noises based on the movements. I named it Music to the dance. As for the movements, technically, teh performer can move in whatever way they want. However, I choose the 2nd Radio Calisthenics For Primary School in China and it’s mandatory for all students from primary schools to high schools to do it every weekday. Personally, I think it’s quite special because not every country will require that. Besides, since I used to do it everyday, I’m familiar and feel more comfortable with it. And the movements are not really complicated, so it’s also easy for me to invite the audience to interact with me. When a movement is detected, the noise will be generated.


When I was presenting this idea, I thought I would use Makey Makey to trigger or loop the sound and make interactions with the audience. (For example, when teh performer and the audience touch, a sound will be played.) Besides, I also thought that I will use arduino and Max/MSP because we can easily control teh arduino with Max. However, after talking to professor Antonius, I decided to use my webcam and Max. I think it will be interesting to have no wires connected and I am amazed by what a webcam and a software can create together.


  1. I wanted to open and close my webcam with Max. Therefore, I used the object jit.grab to create a 320*240 matrice to contain my camera. The user is able to create any size of the container as they want. Then I passed two messages — open and close — to tell Max to either open or close the webcam.
  2. In order to detect the motion, I need to cut my whole camera into different pieces. I used git.scissors and set both rows and columns to 3, so I get a 3*3 = 9 pieces of camera. The user can also cut into any number of pieces as they want by simply changing the rows and columns.
  3. As for the motion detection, Jitter has an object called jit.op which can tell the difference between two matrices. This is really important because it will report the differences to the user. I forgot to use it the first time I tried it, so I could only get the real ARGB value instead of the differences.
  4. For more accurate detecting, I choose to use jit.3m, an object which can report the minimum, mean and maximum ARGB value (the order is ARGB as well). Then, I used unpack object to get the seperate ARGB mean value as a float number.
  5. If we directly use the number we get from the unpack object to generate noise, we will quickly find that the whole sound is too noisy, because as long as there is a motion, no matter how minor it is, the noise will be generated. Therefore, I used > object to filter out some small numbers so that the performer can only trigger the noise when they intend to.
  6. Instead of using the sound that I recorded, I used cycle, saw, tri and rect as different oscillators to generate different noise. At the beginning, I also used the keyslider to make random piano keys according to different movement. However, during the rehearsal, I was suggested to clean the sound a little bit and put the sound in more diverse areas so that when I moved, I could hear a clearly different sound. Therefore, I removed the keyslider.
  7.   I choose the 2nd, 4th, 6th, 8th pieces of the camera (shown in the image below). And I didn’t attach any oscillator to the center piece because basically every movement will trigger that part and that will make the whole sound a bit noisy.

Self-reflection of the NIME concert:

To be honest, I was really nervous before the concert because I’m afraid that 4 classes is not enough for me to develop an instrument that I can perform at the show. However, during the rehearsal on Thursday, my classmates and professors and fellows sincerely brought up suggestions for me. At the beginning, I planned to face the audience and make some dance movements. Because I am clear that I divided the whole camera into several parts, I had this thought in mind that I have to move within the range and make all four noises that I have. The problem with this thought is that I would keep looking at the camera and feel fear of moving freely. Therefore, Professor Antonius suggested me to turn around and not face the audience. This inspired me to use the blind-fold to cover my eyes so that I won’t be able to see anything. Also, I could just grab audience and give them the blindfold. In my opinion, this is the crucial part of my performance.

Actually, by covering my eyes, I get rid of two boundaries in my mind. The first is the actual boundary of my camera. Without this restriction, I won’t feel fear of getting out of the camera or not making every sound that I have anymore while in the rehearsal I kept looking at the computer or camera and didn’t have eye contact with my audience. The second boundary is the one between me as a performer and my audience. In the rehearsal, I was worried that the audience would refuse my invitation to try the instrument. However, with my eyes covered, I can grab anyone I touch and people definitely won’t criticize me because I’m “blind”! As a new NIMEr, I used to think that if I want my performance to stand out, I had to include as much technology as possible. However, after this concert, I start to think that we shouldn’t be limited to technology. And a performance is not necessarily less fun without a lot fancy techniques. This can be a really important lesson for me as an IMA student.

When reflecting my final performance, I remembered that Professor Antonius said that the technology is not a problem because we can get help from all the fellows. However, the interaction is more important because it gives the project a meaning and it determines who is this project built for and how other people can enjoy it as well. Therefore, although this simple piece of instrument (or maybe it couldn’t even be called as in instrument) is not that technological, I’m still proud of myself that my performance brought the audience fun. In general, as an IMA student, I always wonder what job i can get after graduation and which direction I should focus on. I’m grateful that this class gave me so much fun and confidence to make sure that I want to do visual designs and more things about music! In a word, this may be the most productive IMA semester I have: I built my Nature of Code project to be a sound visualizer and I created several logos for NIME organization (check this link to see all the logos) and finally I had my NIME instrument. It’s really interesting to see how a simple idea can generate so many results — I’m looking forward to more!


Source Code: (Some how I couldn’t copy and paste the patch so…)

NIME – In Class Assignment (Max / MSP)


First, I called the MIDI object and connected it to the noteout object. Next, I created a toggle and attached it to the input part of a metronome which I set up to be at 150 milliseconds. I connected the metronome a random object, and then connected this one to the makenote object and gave it a velocity of 400 milliseconds and a duration of 100. I lastly connected the makeout object to a noteout object.

Creating Sound: 

I first created an int number and a float number. I connected the int to the right input of a cycle~ object, which controls the frequency, and the float number to the left input of cycle~, for the number of phases. Then I created a gain slider and connected the cycle~ to it. Lastly, I connected the scaled output of the gain slider to both of the inputs of the ezdac and to the X display of the sound visualizer.

Random, Operations and Sound:

I created a toggle and connected it to the input of the metro object, which i set at 150 milliseconds. The metro object was then connected to a bang button, which was attached to a random object of 60. This one was then connected to an int number and to a adding object, which was also connected to another int number attached to the right inout of the midi. The int number previously mentioned was connected to an adding function, which was linked to an int number attached to the left input of the midi. The left output of the midi was connected to an int number connected to the makenote object which had a velocity of 80 milliseconds and a duration of 100. The left output of the makenote object was linked to the left input of a noteout object, and the right output was connected to the middle inout of noteout. Lastly, the right output of the midi was connected to the middle input of the makenote object.

Playing a Sound File:

First I created a toggle and attached it to a sfplay~ object. The left input of this object was also attached to three messages, open, loop 0, and loop1. The left output of the sfplay~ was attacched to a gain slider, which was then connected to the speaker ezdac objet on both inputs.

Buffer and Groove:

I created a message called read and connected it to the left input of a buffer~ object. The I created an int number and connected to a sig~ object. This object was then connected to a groove~ obect which called the same audio file called in the buffer~ object. I also created another message and to loop the file and linked it to the left inout of the groove~. Lastly, I created an ezdac object and connected it to the left output of the groove~.


I connected power and ground from the Arduino to the breadboard. Then I placed two potentiometers on the breadboard and connected their middle pins to an analog input pin, another one to ground, and the other one to power. Then I placed the switch on the breadboard and connect one of their pins to a digital input pin and another one to a 10k resistor which was also connected to ground.  I was not able to patch the max code from Arduino to Max on my laptop so I had to use my friends laptop. However, a picture of the Arduino and a video of the result can be seen below:

Difficulties: One of the biggest issues I had when doing these exercises was making sure I was using the right object, because some of them look very similar so I made several mistakes just because I was not paying enough attention to every detail.

[NIME] Using Max Console Yuntian Ye

First I use a typical programming test to the language, print (‘Hello World’) to see the logic.

In Max, the procedure is from up to down and from right to left.

When yo push the bang bottom, the program will start once to print.

I try different object to have different effect. Metro can control the speed of Bang being pushed; random generate random number; makenote take the random number as the pitch and it also control the velocity and duration; outout can play the note repeatedly.

Furthermore, you can add keyboard to see what key is being pressed.

Max can also accept external audio source and play. The gain bar can provide user a clear visualization of controlling volume. Loop 0 means play once and loop 1 means play again and again. The loudspeaker button is activated when receiving wave value, I suppose, different from noteout receiving number value.

For a musician, it is not enough to merely control music. Music visualization is also important.

With Minim library of processing, you can have the different values from the music expressed by by actual waving and rectangles.


[NIME]Week3: MSP(Cyndi)

Date: Nov.30th

Name: Cyndi

Instructor: Antonius

Goal: To learn how to use MSP


Last class, we learned how to use the software MSP to generate different sound.

First, we learned how to generate random sound. I tried to put a really large number in the random function at the beginning, but some of the sounds are not generated. With the help of professor Antonius, I  learned that the range of the random is 128. Every value represents different pitches.

Then, we learned how to play a local song and control its volume using MSP. I learned that the buffer is used to store the file.

At last, I tried to explore MSP with Arduino. I found out my port is b but didn’t have enough time to practice it. I will spend more time practicing the interaction between MSP and Arduino.


This software is very very useful and I can generate many ideal sounds by using it. I am thinking about applying it to my final instrument. However, extra attention is needed when connecting those ports together.



NIME performance 2 (Ronan)

This is teh video for my second performence.

In general, I think I’m more confortable with performing on stage than the first time.

Some improvements which could have been done are:

first, telling the professor that I will sing a little bit in the sound check so that we can have everything prepared well ahead of time.

second, I showed two different sounds with one single instrument, but the transition between them is not very good. Therefore, I think it will be better if I switched the order of these two sounds and showed audience the second sound (piano like) first. Or even make a switch on the instrument instead of working on my computer.