(Rudi) Final Project (Antonia)

Project Name:”Meet you in the heaven”

Project Producer:Anna Li(Antonia Li)/ Chenchen

Project Description:

For the final project, we created an interactive game within a story, which is about a man rebuilding his wife. The context of this story is that the husband treated his wife bad and regretted after her death, and then an angel came to him, saying that she could help him rebuild his wife. So the whole game is the man’s adventure in the heaven to gather his wife’s soul and body to make her reborn.

There are two parts of the game. The physical part is made of a small map. There is a movable human figure for user to move, and several little golden circle on the map where user can put the figure on and trigger the plot.

And on the computer screen, the plot will be showed. There will be conversations, or little games.

The Whole Process of the Project :


In our mid-term project, I and Chenchen made a visual story where the users can read a story while the scenes in the story were physically made, and the user has to find the right time line and choose the right scene to the story continue to happen. But this project has a vital defect that nearly all the users fail to understand what they should do and get lost. However, both the professors and my classmates think it’s a good idea to make a story with physical scene. So in the final project, we decided to continue and improve our mid-term project, add more new factors on it, and make it more understandable.

So we decided to work on those defects our mid-term project has, and we decided to make three improvements:

1.Users themselves can choose which scene to go next.

2.There will be three endings, and the story will not be linear anymore.

3.To make it more interactive, we decided to create little riddle games.

So basically this is our new plan, and then we began to work on that.


Stage 1. time planning

Actually at first we were quite confused because the project is quite big and we had no idea where to begin. So in the first week, we were just doing some random things. Like I found some figures or backgrounds that might be used in the game, and edited them in the Photoshop like:

And Chenchen and I both began to think about how to write the code for those little games like one part called “the potion part”. However, Leon came to us that we shouldn’t do those very detailed thing at first. What we need is to write down every thing we need to do, and make a time table for each thing. Do the general plan.

So we made our flowchart and our own time table.


So then, we began to develop everything by the timetable and our flow chart.

And my part is mainly about the plot writing, and turn them into code; backgrounds, figures combing, circuit soldering and connecting, and physical part making.

Stage 2. Plot writing

At first, I Planned to write a very complicate story with interesting plot and deep reflection to our real world. However, I found out that it would take too many words for such a story and the user might lose their patience for such a long plot.

<The original version>

So I decided to simplify it. I talked with Leon about it because I found it too hard to tell a story in less than 100 words. But then he suggested me that not every story needs to be told in words. We could use images to tell stories, which is more interesting and good for time saving. So with this idea, I decided to make an animation that could easily shows the story at the beginning.

So I began to draw. I used the figure on the internet and edited them in the Photoshop. And then used the mouse to draw different position of them, and saved every picture.

And then I used iMovie and Final Cut Pro to combine them, adding the music. And it turned out that I like this small movie very much and so do my friends. However, after I talked to Rudi and Leon, they found out that one-minute video is too long. So I managed to cut it to 37 seconds, which was the best I could do. I was not very willing to cut it too short because I thought it was telling a story, and if we cut most of it, it would not make sense and will lose the most interesting parts. In my views, a video at the beginning of the game could make the people calm down and gradually immerse in the atmosphere of this game. 🙂

And here is the 37s video(which later Chenchen cut it to 19s):


Stage 3.Backgrounds, figures combing

I chose very difficult backgrounds and animation figures at first (as above), but Chenchen reminded me that the flower in the game she made were quite simple which could not match those complicate figures. And then Leon told me about the importance of consistency in a game, which the figures, backgrounds, and other elements should be in similar styles. So I have to use more simple figure and backgrounds.

So I decided to continue using the figures from mid-term projects, but in a different ways. And I found and drew some backgrounds in the Photoshop using the simple gradient ramp, which turned out to work quite well.

So it would be much more consistent than the preview ones.

And then I combined them together, and for the first time, I learned how to create function in the Processing with ease. And my understanding of functions in Processing has been improved much more during the coding process. And then I found out that most difficult thing is to calculate the location of the text and how much time users need to read them. And it’s totally a math thing.

Stage 4.Circuit soldering and connecting

After I have turned all the story part into code, I gave them to Chenchen and she began to work on the whole structure of the coding part, and I began to work on the circuit connecting part.

In the project, we planned to use conductive tape as switches. It is disconnected at first, and when user put the human figure on it, it will become into access. So my job is to think about how to make it more beautiful as well as easy to control. There are seven parts in the physical map, one start point, two Angels, two little games, and two Holy Lands (the waterfall and the fire), and there are 11switches. So I began to think about how to make each of them.

For the Angels, I chose to make it more beautifully. So I chose to make a heart shape switch on it. And when the user put the figure on it, it looks like that the Angel is lifting him with her hands. And this might make it more sacred.

And for the other parts, I made them exactly the same to make it more consistent and easy for users to understand. So I cut several circle of the same sizes, and attached the conductive tape on it. And then on the back of those circles, I soldered two short wires to make it through the paperboard and attached them on the back of the board.

Then the most difficult part is to connect all those short wires to the breadboard. And I did that with Chenchen together. We tried our best to solder those wires together, and make sure that they would disconnect.

Stage 5.Physical part making.

After we connected all the circuit, I began to work on the surface of the board, the map. I bought several elements on Taobao and also made some on my own. Like the flower, or the bottles for the  potion. And this was the most fun part of the whole project, “it’s time when all the project becomes beautiful”, as Luise said. So I tried my best to make it beautiful. I drew the two Angels on the board, and made those flowers.

Stage 6. the Outcome

We videoed our outcome by recording several small clips:

Click:The accelerated beginning part

Click: Potion Part wrong twice

Click: Potion Part wrong once

The flower part:



It was quite a long time preparing this project, and I’ve learned a lot.


Now I know that every elements in a project should be similar so that the consistency of the whole project would not be damaged. And one project should only have one style, it could be creepy, cute, but can’t be both at the same time.

2.time managing

With the help of Leon, we learned that for a big project like this, we should make a time table and clearly know about what we should do and what we should do next. That’s the only we to insure that we would not miss something or fail to complete it on time.


I worked with Chenchen this time as well as last time. And Rudi, Leon and other fellow have come to talk with us. I found it was actually not easy in a project to collaborate with other people, because every one has their own thoughts about the project and sometimes are very different. So it’s quite difficult to really decide. And fortunately, I and Chenchen worked it out, and at least 80% of time we reached an agreement. And it was totally fine to have different opinions, that is the way to make our project more colorful.


I wrote the plot coding part and gave a basic logic of potion parts, which was quite meaningful for me because I found it very helpful for my understanding of coding. Coding has been a large obstacle for me, but doing all this helps me understand it more and have more control over it.

5.circuit connecting and managing

I found out that it’s actually very difficult to manage the circuit, and very hard to make it clean and tidy. And it’s a big headache for me. But fortunately I worked it out, although took a long time. And experience told me that next time the management of the circuit should be considered at the very beginning or every thing could be a big mess later.




(Rudi)Assignment for class 25

After reading the assigned materials and checking in detail Real Life Applicationsand Art Installations sections in our slides, choose one example of Computer Vision. Research about that example and post in your blog your findings. Be specific about which tools you think were used and which challenges the developers faced. In your response, clarify why you chose it and how does it relate to your own experiences with Computer Vision. Due December 7th.

What I choose: 

Government <> Privacy

Why I choose it: 

Why I choose the monitoring system set by government because I watched a famous TV series called Person of Interest years ago, which really impressed me and then I began to think about the problems of surveillance and personal privacy. We all know that government set the surveillance system to monitor the bad guy and to detect the potential threat. However, this process also threatens the normal people’s privacy and the technology might be used with dirty hands to commit more serious crime. So it’s really difficult to balance the public safety and personal privacy. And people keep arguing about that and can’t decide which is more important.

which tools I think were used:

First, I think there must be thousands of camera being used to get everybody’s faces. And then there should be face tracker technology like faceOSC to distinguish real human faces from other things. After they getting those faces, there should be some kind of face recognition system that could analyze the features of faces it detects. Like the location of eyes, the size of the nose. And then, it can compare them with the face information in the database and decide which person that is.

which challenges the developers faced:

  1. It’s a little bit difficult for machine to decide which face is from real human and which is from paper(like posters or masks). So if a person wear a mask, the computer might recognize him as another person.
  2. The differences between different faces might tiny and difficult to detect. Especially when some one is moving, it’s a lot more difficult for camera to capture the exact features of his face.
  3. People’s faces might be very different when they have different facial expression.
  4. Moral issue: people will doubt if it’s an invasion of privacy when their faces are recognized and recorded.

(Rudi)(Anna)Lab 10- Review

Lab 10 – Review

In the past ten weeks you have learned the basics of electronics and the fundamentals of programming. Now it’s time to review your skills.

Meet first in Room 823 to sign up for one of the workshops:

  • Soldering and Better Circuit Design – Room 818
  • Serial Communication – Room 824
  • Object Oriented Programming – Room 823

My choice:

I chose to learn soldering, because I thought it is a very useful skill, and I could apply that on my projects. It would be much more solid and could make the project more beautiful.


It was my first time to solder things so I was quite afraid of the high temperature at first and afraid to make any move. But with the help of Leon, I had my first try, and then I found out that it was pretty fun. But my skill was not good at all so the first two connecting seemed quite ugly and unstable. But then I got a little more familiar with it, and become more relaxed and the whole process was quite enjoyable.



How would I introduce soldering to other people:

Soldering is a method to connect two metal products. And the theory is that the melting point of metal Stannum is much lower than others’ so we could melt Stannum at a low temperature while other metal is still solid. Then the melting Stannum can become any form we want, and when it is cooling again, it is solid and it becomes the connection between two other metals.


(Rudi)(Anna)Lab 11 – Media Controller

Lab 11 – Media Controller

Now that you know how to work with images, video, webcam and audio in Processing, your assignment is to create a Processing sketch that controls media elements (images, videos and audio) using a physical controller made with Arduino.

My Idea

Because I like anything related to videos, so I chose to insert a video in processing. And when I thought about what sensor to use, I saw a light sensor in the kit, and it occurred to me that I could use a light sensor and a video about light to create an interactive stuff, so I began to search the video online.


Then I used Baidu searching for a candle light. But I only found a gif, and when I downloaded it, it was not continuous any more. So I came up with an idea to use quicktime to screenshot it to make it a short video, and finally it worked out.

Click me to see the video

And then I began to work on the physical part and I connected the circuit. The circuit of light sensor is quite simple.

And the difficult part for me is the coding part, first I worked on the Arduino Part and it could be seen that I made a mistake at first. I write both “Serial.write” and “Serial.println”, so it led to mistakes in processing later. And it was found out by Luise, and then I knew that we can only write one of these two things, or processing part would get confused.

Then the processing part:



So basically, I made a small project which if one sheds light on the light sensor, the candle on the screen will light up too. I like the idea very much, and I was thinking about using it on my final project. That would be interesting.

Click me to see the video for results


(Rudi)Lab 9 – 3D Modeling (Anna)


Using Tinkercad, design a 3D model of a wearable device, a game controller or a mechanism (for a component) that utilizes at least one of the following components or equipment.

My Plan

So I chose to make a wearable device with a moisture sensor in it, so the man can detect the moisture everywhere he goes!(Haven’t figure out why a man would want or need to do so)

I decided to make wristband, and the sensor can be fixed on it, so I put three elements on it to help fix it. And during the process, I used caliper to measure the exact length and width of the sensor, so I can make sure that it can be fit in the wristband.

Then it looks like this.

<<where the sensor is fixed

Then I added some words in the back to make it more.. beautiful.



It’s kind of difficult for me when I first use this, and kind of hard to make everything in the proper position and proper size. But it was fun building 3D models and I think this is quite useful and more convenient for us to build something that exists in our mind.

(Rudi)Laser cutting: Illustrator(Anna)

So in this class, I followed the instruction in the video and made a symbol for IMA NYUSH.

It’s very fun and useful to learn about illustrator and I finally got a chance to learn something about it and I found it’s very different from Photoshop, especially when our aim is to make our symbols laser cut. It’s a very convenient way! Much easier than user knife to cut everything by ourselves.

Click me to see my Symbole:ANNA1

(Rudi)Final Project Essay (Anna)

Definition of Interaction

Interaction for me is that the project can react to user’s movement, and gives him a series response, raising new question for user so he can continue acting to the project. The user’s reaction will influence the project’s processing and the project’s respond will influence user’s choice.

My Final Project Ideas

In the final project, we decided to continue and improve our mid-term project, in which we told a story about a man abusing his loving wife. And this time, we want to continue the story, where an angel come to that regretful man and tell them that his wife can reborn if he goes to the heaven and help gather the piece for her rebuilding. So basically, the whole game is the man’s adventure in the heaven.

The heaven will be made physically. We will make a map with four parts, and each part has a God figure and some surroundings. The main character will be a small human figure, and user can move it with their own hands. The game will begin at one specific part, and then by the storyline, the user can make his own choice which part he decide to go to next. And each part will have a specific position where user can put their small figure, and we will use pressure sensor to sense it and trigger the conversation and the plot, which will be showed on the Processing part. It is worth mentioning that when the user put the small figure down, the God and the surrounding of that part will have some physical movement, like the angel will move her wings, and the flowers will light up or something.

The story will have three endings. The first one is the happy ending where the man and his wife survive and go back to their own world. The second one is the normal ending where the man sacrifice himself to save his wife and only his wife survive and get back to their own world. The third one is the bad ending where neither of them survive. And which ending the user will get depends on the choices he make or whether he solve some puzzles in time.

We want to make the story more interesting in this way, where the user can move the figure with their own hands and put it any point they want, and they can experience different plot and different result. So actually the user’s movement will lead the story line and the story itself can influence the way users think.

(Rudi)Anna-Lab 8- Drawing Machines


We will be creating drawing machines by using an H-bridge to control stepper motors attached to mechanical arms. Individually assemble the circuit using the SN75440NE IC and the pre-installed Arduino Stepper Library to control one stepper motor.

Part 1: Build the following circuit to control the stepper.

This must be the most difficult part. So many wires and so difficult the circuit is! Although I had double checked the circuit before connect it to the power, I still got it wrong and burnt my Arduino Uno (which was actually borrowed from the equipment room). I got it wrong because I directly connected the 12V power to the Arduino uno, and you can see from the following picture.

The second time I connected it with a new Arduino (which was borrowed from the Equipment room again), I did it with great care and have a fellow help me to check it, and finally I did it.

Click me:IMG_2432

Part 2: Use your potentiometer and the MotorKnob example to control your motor. Please note that 42STH33-0404AC stepper motor is a 200 step motor. You can use the function map() in order to match the movement of the knob with the rotation of the motor.

So then I added a potentiometer on the breadboard. But I was kind suffered about the code, because I was not so familiar with the map(), but after a little searching, I finally got it right.

Click me:IMG_24331

Part 3: Write a sketch on Processing that send values to Arduino. Replace the potentiometer by using the values from Processing to control the motor.

This took me quite some time because when I finished the first two tasks the class was over. So I had to return everything and borrowed them again to continue.I used a very simple sketch and sent the variable to the Arduino. The principle is that:when you pushed the mouse, the motor spin clockwise, and vice versa.

Click me:IMG_2436

Part 4: Then, find another person to work with. Combine your parts into a mechanical arm that can hold a marker —see the picture below. Use processing to control the movement of your motor, and make adjustments with your partner.You have just created a drawing machine!

Since the class was over then, so I changed my partner which was chenchen. We both use our own function to control the motor and make the drawing machine to draw some random thing. And I found that the pic they draw were much more neatly than I thought, which is awesome. But I think that it might be difficult to make them to draw some real picture and it must need a lot calculation.

Click me:IMG_4169


(Rudi)Anna-User Testing(My projected to have been tested)

What’s my project:

“Back to Past”:This is project composes of two parts, physical and digital. The physical part is  a movable pointer, two buttons, a half-circle board with small scene models, and those scenes come from the story told in the computer screen, which is the digital part. The users have to move the buttons to move the pointer to the different scene and they can find out the story for that scene showing on the computer.

How my user interact with my project:

Because my project is telling a story and takes about 3-4 minutes, so usually there was one person actually interacting with the project and others watching, which might be a good thing or bad thing. It is good because that actually everyone can join the story sharing process, and the bad thing is that one story takes a little bit long and the audience might get bored.

And there is one serious problem is that every one coming to test our project became confused when they have to spin the pointer to make the story continue happening. Although we writes every rule clearly in the story (We suppose that our user is the main character of the story and what he has to do should follow the instruction that has been assigned to the main character in the story). But it turned out that few of people will read the instructions carefully so they would get confused. This is the point that we fail to consider about because we never thought about that users might lose patience to read every word on screen with great cautious.


Suggestions and Comments:

  1. Marcela suggested that we should rotate the whole paper board rather than the pointer, and hide other scenes and only show one scene at a time and it could be more mysterious.I think that she gives us this suggestion might be that the all five scene models on the paperboard might seem little bit confusing to people, and it’s too obvious and may leak our intentions.
  2. Luise suggested that we should give our story different endings so that we could have more interactions with users. Users can chose how the story ends by choosing some critical point in the story, and they can actually control how the story goes. And I think he gave us this suggestion is that the simple one line story might seem boring for audience and they can barely feel nothing after watching it; plus, the interaction between users and the projects is too simple and lack of meaning.


I think we should add some critical moment in our story and make it more attractive. We should try our best to impress people in the limited time. Different endings, rotating paperboard, or any other forms, we should make people feel it’s interesting and want to continue the story.We might enrich the story and change the form we tell it, and change the way how users interact with it.

(Rudi)Anna-User Testing(The one I tested)

In the Lab 7th, I tested almost every one’s project and the one which left me with great impression is the project about dandelion(Made by Mary Wang). In this project, there is a “dandelion” in a small bottle, and a “dandelion” on the computer screen, and when you lift the “dandelion” up, and blow towards it, the seeds of the dandelion on the screen will fly freely.

I think this is very cute and make people feel warm. The scene when those white seeds fly is so beautiful and makes people relaxed.

Questions: What would happen if we blow out all of the seed?Will the seed gone with the wind and breed else where?

Suggestions: 1.The restart of the project has to click the keyboard so the seeds can fly back to the dandelion, which might destroy the beauty of the whole process, so the maker can fix like this:when we put back the dandelion into the bottle the dandelion on the screen can be automatically restart.

2.It might be a little bit simple that the interaction process between the user and the project might be too short, so the maker might create a specific scene or background story for it.