Interaction Lab, Final Project, Yun-Jui Hsieh (Jerry), Professor Mikesell

Jerry Hsieh

Prof. Daniel Mikesell

Final Project

Project Title: Introduction to Taiwan

For the final project of the interaction lab, initially, I wanted to make something for my hometown, Taiwan. Therefore I came up with the idea of making a game that introduces Taiwanese foods and scenery spots.

Initially, I wanted to make a racing game in the form of monopoly, however, I soon found out it was too challenging for me. I started from making the background screen and multiple choice screens separately.

For the background screen, I inserted a Taiwanese food map and drew several ellipses as the stations. I used a Taiwanese bear figure and treated it as the moving figure in the game.


Screen Shot 2017-05-19 at 9.23.49 AM

For the multiple choice questions, I inserted three images for each one, and I listed them either vertical or horizontally. Questions are connected to their respective stations.

Sample multiple choice screens:

Screen Shot 2017-05-18 at 3.23.57 PM Screen Shot 2017-05-18 at 3.24.19 PM Screen Shot 2017-05-18 at 3.24.32 PM Screen Shot 2017-05-18 at 3.24.47 PM

After finishing the screens, I started to make a dice. Since the Arduino board would not be wireless, I decided to make an animated dice. Therefore, I used an accelerometer. Every time when the number of z-axis was above 0.8, it would activate the dice function and show the random value on the screen. Also, I tried to make the animation of the bear figure. At first, I just had four stations, so I tried to use the basic way to

Also, I tried to make the animation of the bear figure according to the dice number. At first, I just had four stations, so I tried the most basic way to move it. I created several functions, such as “TaipeiToHualien” and “KaoxiongToTaichung.” However, if I created movement by this way, I would need a lot of functions and if statements, therefore I asked for help.

A professor told me that I should turn my stations, as well as my questions, into objects, so I started doing some research about object and class. After asking my classmate about the concept of object, I soon understand the way to make a full use of its function. Here is the sample code I wrote for each station. The first five are about the stations, and the rest are about the questions. To activate them, I set a “display” function for the former ones and “ask” for the latter ones.


Stop(float xpos_, float ypos_, float sizeX_, float sizeY_, String name_, String Title_, PImage firstImage_, PImage secondImage_, PImage thirdImage_, int option_, color bg_, String foodname1_, String foodname2_, String foodname3_)


Taipei = new Stop(620, 75, 200, 100, “1.Taipei”, “Which is not in Taipei?”, SC, CKC, River, 4, bg, “101”, “Chiang Kai-Shek Memorial Hall”, “Love River”);

The only question left is the animation of the bear figure. The professor helps me set an integer named “index,” which is related to the dice number, and control to the position of the bear by index. Index equals to 0 originally, so the bear stays at Taipei, the first stop. If the dice number is 3 when first throwing the dice, then the bear will be moved to the fourth station, Tainan. However, I gave up the animation effect because it is way more complex.

Sample code:

if (index == 0 && stateval==0) { // tapiei

At the same time, I also use the index to control the questions. For example, in my code I wrote:

if (index == 0) { //play Taipei game

The code was not 100% correct, but it somehow worked.

In addition, I created another class that includes images of dice number. The dice numbers include merely 1,2, and 3. After revision on the order of codes, my project worked very well.

After inserting those images, my final project function like this:

Furthermore, I gave up using the accelerometer at last because it didn’t match with my game. I came up the idea of using a flying lantern, which is popular in north Taiwan, as the controller. I also added a little design with the lantern. I attached a LED light to it, and every time when it is lifted up, it will be lit up, which is the same thing as the real flying lantern (after firing its bottom, it will fly). I ultimately used a light sensor to detect the movement, but finally discovered that the light sensor is not a good choice because the intense of light varies everywhere and in different angles.

Final Code


The first important thing I have learned from this project is the strength of “object.” Since both the appearance of the stations and the multiple choice questions are all connected with their respective names, the object function is very helpful since it combines information related to same stuff together. I can put everything about the object into the constructor and set different functions to display the information. Also, using object function helps organize the code. It makes the code understandable and neat.

Second, I learned more about boolean function. A game includes a lot of “yes or no”s, so the boolean function is much more helpful than if statements sometimes. Therefore, I use many booleans to call other function, which makes my code a lot easier.

Third, I have to be careful of the game logic, which bothered me a lot initially because I did not know what exactly I wanted to show. However, after paper prototyping my project and thinking through it carefully, I finally figured out the order I wanted and the visual I wanted to show. Prototyping is very important when constructing a game, and the game logic and the order of the code is very important.


Although my project is not perfect, I enjoy the process of making it a lot. I really learned a lot about coding from this project, and I believe I will be able to do a better job next time.

Interaction Lab, Lab 12: Media Controller, Yun-Jui Hsieh, Professor Mikesell

Author: Yun-Jui Hsieh (Jerry)

Professor: Daniel Mikesell

Partner: Romola Zhang

Lab Date: May 5th. 2017

Project: Media Controller

For today’s project, we need to use sensors with Arduino to control the images/videos on the computer screen.

I teamed up with Romola. First, we decided to use a light sensor to control the color of an image. We follow the instruction picture of the light sensor and then connected it to the computer. On the Processing side, we imported serial communication to import the number from Arduino. Also, we use PImage and loadImage function to import an image into Processing. Then we use the “tint” function to adjust the color of it. Once the light intensity changes, the color of the picture changes.

After finishing the first one, we wanted to try something new. We downloaded the library of video in Processing, and we used an example from the Processing website of “capture” function. “Capture” function is a function that can open the camera on the computer. After testing the way the function work, we add an “if statement” to control the camera.


Basically, we had practiced most of the things that we had to use for the lab today, so it was not difficult. However, we still learned some new function from the project, such as the ways to import photos, videos, and camera. I am very happy to learn that much knowledge!

Interaction Lab, Week 11: Drawing Machine, Yun-Jui Hsieh, Professor Mikesell

Author: Yun-Jui Hsieh

Instructor: Dan

Date: May 1st.

Project: Drawing Machine

Material used: DFRduino board, cables, breadboard, stepper motor, 12V converter, SH75440NE ic chip

I was absent from the Friday’s session, so I made up the lab by myself with professor Moon.

First, I followed the instruction and opened the example called “.” Then I followed the picture to create the circuit.


After checking all the elements with professor Moon, my circuit looked like this:

Since professor Moon couldn’t find the plastic stuff that I was supposed to use, we used thin plastic wire instead. We then connected a pen to the stepper motor, and then we put a sheet of paper under them.

Initially, it looked like this:

We slightly changed the code:

This is what my circuit looked like:

IMG_4686 IMG_4689 IMG_4688 IMG_4687


Since I was running out of time and did the project without a partner, I didn’t add a potentiometer to my circuit. Therefore, my circuit may look a little bit simple, but I basically finished all the requirements of the project.

This project is very interesting because I learned the way to make my circuit draw a big round circle for me. I will try to add more controller or sensors to my circuit so I can make it more interactive and fun.

Final Project Idea, Yun-Jui Hsieh, Prof. Mikesell

For my final project of interaction lab class, I would like to make another interactive game. To start proposing my own idea, I would like to define “interaction” beforehand.

Interaction means that each object should response to others’ movements and changes. Interaction should be a reciprocal cycle of information transmission instead of a one-way communication. For example, if a person keeps talking to others without listening to others’ idea, he or she is not interacting with others. However, if the person exchanges ideas with others, he or she is involved in interaction. Therefore, interaction includes information output and feedback, and both of them should be strongly related or easily associated. That is, the feedback should give a valid response, which can be easily understood by others, to the information output. For instance, in people’s daily life, red is associated with “no,” “stop,” while green is associated with “yes,” “right.” Therefore, if a interactive machine shows a red sign when it wants to say “yes” and shows green on the other hand, it is not doing good interaction since it may confuse the users. In short, interaction needs information output and feedback, and both of them should be connected.

For my final project, I will make a game that introduces Taiwan. As a native Taiwanese, I always want to introduce my hometown to my friends, so I want to make a game that can make people know more about it. It will be a racing game which involves two people. On the background, it will be a map of Taiwan, and I will create several square blocks as stations. Each person will take turns and throw a dice, and the figures on the screen will be moved to the stations according to the number of the dice. Furthermore, after arriving at each station, they need to answer the questions shown on the screen. If they answer correctly, they will stay, otherwise, they will be sent back to the previous places.

For the interaction part, I will make a physical dice which players can play with. However, since I am not able to use a wireless sensor, there definitely will be cables around the dice, which will make the result unfair. Therefore, I will just attach a vibration sensor to the dice, and once the sensor detects the movement, the screen will show a random number. By doing so, I can make the result fair and interactive at the same time.

I have made a game for my midterm project already, so gaming logic and the usage of images are not unfamiliar to me. Also, I tried using several sensors before, such as tilt sensor, accelerometer, joystick, etc., therefore making a controller is also something that I am able to do. The challenge of the project is that I have to make sure each step will not affect another, and the players can finish the game in the right order.

Interaction Lab, Week 10, 3D Modeling, Yun-Jui Hsieh, Professor Mikesell

Author: Yun-Jui Hsieh (Jerry)

Professor: Dan

Date: Apr. 21th. 2017

Project: Make a wearable device, a game controller or a security device by TinkerCAD

Partner: Maya Wang

Initially, I wanted to make a traditional controller because it can be easily connected with a joystick. However, Maya refused to do that with me because she thought that was not innovative. Then we got the idea of making a helmet with a propeller, which is one of Doraemon’s most famous tools.

We started from drawing a big half sphere as the top part of the helmet, and then we drew another half sphere as the back part of the helmet. We use another sphere to dig a hole, hollowing the middle part of both the half spheres.

We need to attach a stepper motor on the top of the helmet in order to make the propeller move. Therefore we dig another 4*4*4 box-shaped hole there.Screen Shot 2017-04-21 at 11.52.08 AM Screen Shot 2017-04-21 at 11.52.17 AM Screen Shot 2017-04-21 at 11.52.24 AMScreen Shot 2017-04-21 at 11.52.35 AM

Then, we moved on to draw the propeller. We drew two elliptic flat columns and connected them together. We also measured the diameter of the stick on the stepper motor and then created a hollowed column which can fit the stick.

Screen Shot 2017-04-21 at 12.05.44 PM Screen Shot 2017-04-21 at 12.05.58 PM Screen Shot 2017-04-21 at 12.06.15 PM

Finally, we combined this two stuff together, and our final product looks like this:

Screen Shot 2017-04-21 at 12.07.57 PM Screen Shot 2017-04-21 at 12.08.27 PM Screen Shot 2017-04-21 at 12.08.15 PM


Actually, this was not my first time making a 3D model, and I did some practice before this class, so it was not very hard for me to start this project. The biggest problem I faced with was that I couldn’t adjust the size by typing numbers, so it took me a lot of time adjusting the size and the position manually.

Overall, I really enjoyed the process of this project, and I am very happy that I made a complete and nice-looking 3D model.

Interaction Lab, Field Trip to Autodesk. Yun-Jui Hsieh (Jerry), Professor Mikesell

Author: Yun-Jui Hsieh (Jerry)

Professor: Daniel Mikesell

Field Trip Date: Apr. 14th 2017

On Friday, we went to visit Autodesk company near the school. The narrator introduced the company to us, about their products, programs, and their technological development. Autodesk recently established online platform called TinkerCAD, which will be used in our class. The most amazing thing is that the 3D printing technology is able to print out products that are nearly impossible for people to make by their hands, and this technology is even used by commercial brands, such as Under Armour (UA). UA even cooperated with them to produce 3D-printed shoes, which looks pretty cool.

Gate of Autodesk


3D-printed Chair

The one on the left uses the least materials, but it is as stable as the other ones.


3D-printed Shanghai Tower (Large)



3D-printed Chess


3D-printed puzzle chair


3D-printed UA shoes


3D-printed violin


3D-printed animal


3D-printed Penguin


Overall, this field trip is really interesting. It provides me with a chance to take a glance on the high-tech industry, and it was enjoyable to see those impressive products.

Interaction Lab_Week 9_illustrator, Stamp_Yun-Jui Hsieh_Professor Mikesell

Author: Yun-Jui Hsieh (Jerry)

Professor: Dan

Date: Apr. 11th (in-class)

Project: Make my own stamp by using illustrator

Initially, I follow all the steps that are introduced in the clip. I created a 24*18 (inches) artboard as what the instruction says, and drew two circles with “stroke” but without “fill.” I also drew two invisible circles that acted as the baselines of the text, and I type my name, “Jerry Hsieh,” on the top and my school, “NYU Shanghai,” on the bottom. Additionally, I drew a cat figure in the middle, using ellipses, triangles and lines. I wish my stamp to be like this:

Screen Shot 2017-04-13 at 4.25.34 PM

Afterwards, I transformed the texts into shape by using the function “making outline,” so then the whole graph was made up of shapes. I created a new layer and drew a circle which is slightly larger than the outer layer, then my stamp looks like this.

Screen Shot 2017-04-13 at 4.39.41 PM

Something weird happened then, but I didn’t notice it. All the white parts on the stamp were supposed to be red at that time, but the ones inside the cat’s eyes were not. However, I kept on going. I used the “exclude” function to make the shapes united, changed the color to red again and then drew black bound for the stamp. Finally, I reversed the stamp vertically; however, it was not like what it is supposed to be.

Screen Shot 2017-04-13 at 4.49.47 PM Screen Shot 2017-04-13 at 4.56.17 PM

Since I really want to make an actual stamp, I tried another time. This time, I simplified the process, merely typing “Jerry,” my English name, in the middle. I re-did all the process myself, and the final product looks nice.

Screen Shot 2017-04-13 at 5.04.08 PM Screen Shot 2017-04-13 at 5.04.22 PM Screen Shot 2017-04-13 at 5.06.16 PM



Overall, this project is interesting. That was my first time using illustrator, so it was a little bit difficult at first. After using for a while, I then got the logic of this software, so I could do something creative in it.

There is still something I need to figure it out. First, I have to know the reason why my first stamp looks so weird. There must be some functions that didn’t work in the smaller circle. Second, I need to try making a more complex and more interesting product next time.

Interaction Lab Midterm: Yun-Jui Hsieh, Maya Wang, Professor Mikesell

Author: Yun-Jui Hsieh

Parter: Maya

Instructor: Daniel Mikesell

The Pharaoh’s Escape: A Maze Stimulation

Using a joystick and mouse to enjoy the story and escape from the pyramid-like maze on the screen.

After entering the game from Processing, a pharaoh figure will appear on the screen and the player has to click on the start button to continue the game. The screen of the story will show, and the following screen will tell the players about the rules and notes. Then it will take the player to the gaming screen, which has a pyramid-like maze and a movable pharaoh figure, and the player can use the joystick to move the pharaoh figure. Moreover,  the wall will block the figure from crossing the wall, so the player must follow the specific track to finish the game. If the player makes the exit within 100 seconds, then the gaming screen will be changed to success screen which shows a rotating pharaoh and the word “Congrats,” otherwise if will be changed to failure screen that has a restart button, which allows the player to start over again and again.


Final Maze

new pyramid_whole



The Whole Project

Conceptual Development

Initially, I wanted to make an extension of my stupid pet trick. I made a “cupholder for lazy people” for that project, and I wanted to add another servo to make it work better. However, my professor turned this proposal down, saying that the midterm project “is about application, not necessarily about the complexity of coding or the items.” He asked me to think about something interactive and related to art, so I started finding some examples of interactive art online. After viewing several picture online, I got the idea of making a delicate LED pedestrian traffic light, whose human figure can move and walk instead of standing still. However, after deeper consideration, I found three difficult problems of the idea that is nearly impossible for me to conquer. First, it is not interactive enough. Second, the DFRduino board might not be able to process such a complex code. It does not have enough holes that I can connect to. Third, the idea is not new. Although the human figure of pedestrian traffic light in Shanghai does not move, it is “movable” in Taiwan, my hometown, for several years. Since it is not innovative, as well as the lack of skills and equipments, I finally gave up this idea.

Then I started thinking about something that can be both displayed on the computer screen and controlled by buttons. The idea of making a maze suddenly stroke me— I can’t remember where this idea came from— so then I started to search for some images online. The idea was approved by my professor in the second individual meeting (03/22/2017), and he told me that I needed to do “collision detection (by color),” so then I stated working on it.

Technical Development

I chose to make a triangle maze at first because I thought rectangle one was not challenging and creative enough. I finished a triangle maze in the following day (03/23/2017) after computing the positions of the lines. In addition, I even composed a simple story about the maze. (Story: the triangle figure is actually a pyramid. One day, an old pharaoh was accidentally buried in the pyramid during his deep, deep sleep. After he woke up, he found himself buried, so he had to escape from the pyramid within 100 seconds, or the entrance would be closed forever!) Moreover, I drew a pharaoh, which acts as the main character in the game. By the way, the pharaoh is made of several quadrangles and lines.

First Pharaoh


First Maze (Triangular)


After finishing the maze and the movable item, the pharaoh, on Friday (03/24/2017), I started to work on “collision detection.” I worked with Luis, and we tried to use the exact positions of the lines to detect collision. However, since nearly all values (x and y) of the lines are not rational, we couldn’t make effective detection. Therefore, after several hours of struggling, I decided to draw another triangular maze which is made of vertical and horizontal lines. I made another maze on Saturday (03/25/2017).

Second (Final) Maze

new pyramid_whole

However, making collision detection by detecting the exact values was still hard, so I asked Jiwong if I could detect by color as what professor Mikesell suggested. Therefore, she taught me how to use “get” function and some basic ideas about pixel. I used to arrow buttons on my computer then, so we only chose four points (because there are only four directions!) around the pharaoh figure’s mouth to do color detection. Finally we made successful collision detection (03/27/2017).

Collision Detection (The detecting points was lower, so it may be confusing at first)

After making collision detection, I started to think about “game logic.” I needed a beginning as well as a ending screen for the game, and I also needed several buttons that brings players to the story screens, the gaming screen, and the restart screen. I asked Luis and Antonius, and they asked me to create several new functions, such as “Successscreen” and “Failscreen,” so that the game can be actually “ended” and “restarted.” Also, I used “if statement” to create several rectangular buttons and used “text” function to type words like “start,” “go” and “restart” on the screen. By the way, the starting screen has a frightened pharaoh figure whose eyeballs are missing, the success screen has a rotating pharaoh figure with its eyeballs, and the failure screen is also a restart screen, which has a big restart button in the middle (03/28/2017 & 03/29/2017).

Final Code on Processing

So far, the Processing side is ready; however, on the Arduino side, our serial code didn’t work. Several days ago, Maya tried to make a tilt sensor, but it simply didn’t work then. We tried to use buttons next, but first, the direction was always wrong, and second, it was not entertaining enough. Finally, we chose to use joystick instead, trying to make our project more interactive. After several hours of research, Maya finally found out the solution, so the joystick can move the pharaoh figure on the screen.

Maya decorated the joystick box afterwards, using some pictures of scarab and bricks to decorate it. In ancient Egyptian myth, scarab would lead the dead person to be alive again, so it makes more sense since we have to rescue the dying pharaoh in our game. Also, using picture of Egyptian bricks allows people to know more about our project at first side, so that is why we chose to use these two elements to decorate our controller.

Joystick box


Lessons Learned

Collision detection is the most difficult technique I have learned in this project. However, there is still one flaw in my collision detection. Initially, I used arrow keys to control the pharaoh, therefore I used merely four points to detect the lines. They are on the top, down, right, left parts of the pharaoh’s mouth. However, we changed the controller to joystick at last, so the detection points sometimes do not work for it. Since joystick can give two values (e.g. up and left) and the same time, the moving object can move diagonally, thus sometimes it crosses the wall since the is no detecting point on the corners. It works fairly well for the most of the time, but sometimes it crosses the wall accidentally.

After doing this project, I have gained a lot of knowledge about Processing. I learned the importance to put codes in the right loops and right order, and learned the way to use some new, unfamiliar functions, such as “get()” and “class.” The improvement in the ability of coding on Processing is the most valuable things I got from doing this project.

Lab 6_Serial Communication

Author: Yun-Jui Hsieh

Instructor: Dan

Date: Mar 17th. 2017

Project: Exchange data between Processing and Arduino

We did this practice yesterday in class, therefore I had a set of codes for mouse and LED already. I started the class from testing the previous code. I set up the circuit again, and it worked fairly well.

After recording the LED circuit, I decided to change the LED to the buzzer. However, the buzzer didn’t work at first. It kept making clicking sound whenever I clicked the mouse. Therefore, I turned to Moon. He told me that I had to use the tone function [tone (pinMode, Frequency, duration) and delay()]. Then I started to set the tone for my buzzer, and I finally made out a song.

The code:

The result:


Today, I am more familiar with the “Serial” code, and know the way to tell the functions of “Read” and “Write.” Now I know how the two softwares work together, and the logic behind them.

Interaction Lab_Stupid Pet Trick (Jerry)

Author: Yun-Jui Hsieh (Jerry)

Instructor: Dan

Project: Stupid Pet Trick—Lazy People’s Cup Holder

I started my project at Wednesday’s workshop. I wanted to make a cupholder which can rotate, but I had no idea of getting started. Therefore, Antonius came to my aid, telling me that I had to connect a servo to the Arduino. Therefore, I used the small servo in my kit box first, and then use the following code to make it work.

Servo A

void setup() {

void loop() {

else {

Then, I borrowed the distant sensor. I got the wrong ones twice, so I spent one hour finding the correct sensor I should use. Finally, Dan helped me find the right IR distance sensor, so I could move on and finish my circuit.


But I got stuck right away because I didn’t know what code I should use. One fellow came to my help, and he told me to open a file in the library. Therefore, my IR sensor started to work.

The next step was to make my sensor be able to control my servo. Again, I asked a fellow, and he taught how to use “if statement.” We typed he following code:

if (sensorValue >= 50)
//I move the servo and wait
Serial.println(“move “);
else {

Therefore, when my hand moved toward the sensor, the servo rotated. Then I move to the final step–make the device out.

I needed to choose something light because small servo couldn’t hold heavy stuff. I chose sponge first. I carved a hole on one side of the sponge, and then I connected the other end to servo. The first problem was that the sponge was too soft, so the other end would drop when it rotated. I tried to use pin and paper tape to make it fixed, it worked somehow but it wasn’t perfect.

So on Thursday’s class, I changed the material I used. The sponge was replaced by cardboard, and the hole was replaced by iron wire, and the small servo was replaced by big one. I used copper wire to connect the cardboard to the big servo, and use a paper box to make my device taller. It finally worked like this.


Although I only had three days to finish this project, I was still able to accomplish the task. After doing this project, I have gained more insight of code and know more about if statement. It was pretty helpful for me to re-exam what I have learned so far.