【Kinetic Interfaces】Midterm Project – Puppet


The project was initially designed to be a 3d puppet that’s built with  Unreal Engine. However, I encountered plenty difficulties during the process of creating a custom player pawn and noticed that there’s no way for me to finish this in time. As a result, the project switched back to Processing and the main mechanism of the puppet is using push and pop matrix as well as the translation from linear movement to rotation.


Unreal Progress:

To enable leap motion input, the user only needs to simply check the box in the plugins menu. After that, create a game mode BP and set the default player pawn to be leap motion related ones. Then under the world settings, change the game mode to the one game mode BP that’s just created. When hitting the play button, the hands will appear in the game.

one set back that I encountered was the plug in files are not displayed in the countet browser as default. Therefore, most of the production time was spet on trying to create that leap motion player controller pawn with skeleton mesh imbeded from skratch. 

It’s possible to add a cable component to the skeleton mesh in the BP therefore, it’s possible to create strings that tie fingers to the puppet.

Capstone Progress


Chatbot code structure. Robot hand, head printed model. Linkage design.

The hands and head are paid to print on Taobao which saved a lot of time and come with a better quality.

In progress:

Body under construction

To do:

Finish the construction of the body.

Refine the chatting system.

Make the physical linkage structure


The 3D printer (largest) is highly likely to make errors while printing. People that have 3D printing involved in their project should take that into concern.

User Testing – Oracle

User testing Process:

  1. The user was shown with the webpage with a basic instruction of “please say something” on and the rest was up to the user.
  2. The user will go through the conversation built so far
  3. Present the user with the 3D model of the final experience of the project
  4. Ask the user for feedback

The testers: 1 with IMA background, 1 with CS background, 1 with Businesses background


The CS tester was particularly interested in poking with the bugs and the flaw in the chatbot system. The tester deliberately tried not to follow the instructions hiding within the conversation. As a result, the conversation was often trapped into a dead loop. The designed dialogue wasn’t working at all.

The IMA tester somehow played around with the chatbot but soon noticed the flaw in the system as well. The tester suggested that the other potential bot system like Microsoft bot framework that would build keyword substitution detection whereas in rivescript the developer kind of need to write those up manually.

The none-programming background tester was really cooperative and follow the verbal guidelines of the bot carefully. However, the tester found out the total time of the conversation was too short to have a complete experience of interaction.

General Feedback:

The bot system needs further development to handle a broad and complete conversation with the user. The speech recognition and the vocal feedback from the bot were generally working well and acceptable. The appearance of the bot was consistent with the impression of the chatting system. The user also suggested that the insult from the robot could potentially be more radical.

Next Step:

The urgent issue for current stage is to build the physical installation of the project since the chatting system is working with no lethal error. The remaining coding part of the project now is to refine the chatting system and further develop that. The models should get shipped for 3d printing asap. The inner linkage mechanism should get done within this week.

Video Game Project – Tier


Project File:


The level is under /content/level/tier


The layout of the project

The initial visual design direction for Tier was a 3D sidescrowller game. Due to the limited time of production, the prototype went for a pixel 2D sidescrowller. The game was constructed based on the 2D Platformer kit which is available for purchase on Unreal store.

Demo screenshot

The major change I made based on the BP that came with the kit were replacing the sprite animation and rewrote several functions in the event graph. I did the key frames for the cat’s animation and the sprite human that’s shown in the image above was a picture found online with the character from Undertale. If time permitted, this should haven been swapped with original pixel art.

Object interaction with state checking

The human BP comes with different characters. Some of them are good and some are bad. When the player’s collision starts overlap with the human collision, the system will check if this human is interactable first, then check if this individual is good or bad. If the human is good, when pressing “E” the player will receive one food and the health bar goes up by one heart, and vise  versa.

Tile map

The tile map editor is really convenient for people to create level design in the game engine. With the later function, the user can easily adjust the properties of the components on the map.

Potential directions to go from here:

This version of the game is really a pre alpha version and most of the elements that I want to show wasn’t able to be done whinin the 2 weeks of production time. If there’s a possibility to further develop this, this first technical part to do is a widget interaction interface for the player to chose how they are gonna approach the human in the game. Moreover, it is also crucial to develop more human characters in the game so that the theme of the game is focused on exploring stories and life of the individuals as an animal, which we would probably ignore in daily life as a human being.

Game Analysis: Prune (Krom)

“prune game”的图片搜索结果

Prune is a 2D mobile game developed by Polyculture and released in 2015. The game won the App Store Best of 2015 award. The overall visual design of Prune is very artistic and concise. The theme color is black, red, and beige. The game took a flat visual style and relaxing interactive music.

“prune game”的图片搜索结果

The objective of the game is to have enough flower blooming from your tree. At the start of the game, the player plant down the tree and the branches will start to grow. There’s a limit on the amount of the sub-branches and the flower will only bloom when the sub-branches are under the sunlight. New branches will not grow from the cut spot. As a result, if the player does not cut away the branches, the tree will eventually stop growing and cannot reach the sunlight. On the other hand, if the player did too much cutting, the tree will die as well. Meanwhile, there are two kinds of blocking mechanism in Prune. The black sphere will break the branch and the red sphere will infect the tree. When the branch touches the black sphere, it will fray off and will no longer grow anymore. If getting intersected with the red sphere, the branch will turn red as well. All the flowers on the red branch will wither away and the infection will spread backway from the tip of the branch to the root. Therefore, the player needs to avoid touching the red sphere. There’s also another type of sphere which is blue. The blue sphere will boost the growth of the tree like fertilizer. However, be aware that this will also boost the speed of the branches that are growing toward the red sphere. When the sub-branches reach the sunlight, flowers will bloom from the branch with music going on in the background and when there’s enough flower, the camera will zoom away, indicating the end of the level. The player can regrow as many trees as they want. Therefore, there isn’t really a losing state in Prune.相关图片

The visual of Prune is very artistic and satisfying. The art style is consistent and works well with the game mechanism. One major factor that makes me consider it to be an art game is that there’s no losing in Prune. In regular games, there is usually a losing state either simply ends the game or contribute to the last-man-standing rule. However, in Prune, it is more about experiencing the cutting branches and searching for light.“prune game”的图片搜索结果

The only mechanism in Prune is trimming branch. This reminds me of the art of ikebana, whereas the performer prunes the flower and reassemble them in the vase or other containers. The process of trimming is very artistic and embrace the feeling of Zen. Meanwhile, the interaction of cutting old branches to get new branches reminds me of an old saying. Give to get. You need to pay the effort, spend the time, under some circumstance, you even have to give up the things you love to move forward, to create the better future. The simulation of abandoning old branches is doing a good job conveying this information. Meanwhile, trimming the infected branches for me represents that people need to keep what is good and discard the dross. When you observe the evilness in your behavior, it is very important to correct those behaviors and make sure not to do it again. Otherwise, it will lead to worse action and corrupt one’s soul in the future.

【Kinetic Interfaces】Week 4 Pixel

The effect that I want to make is when the user clicks the mouse, the small pixels moves in different directions and the image dissolves.

However, I got a problem when trying to use the webcam. I googled it and it seems like the java version isn’t compatible with the filebot. I tried to reinstalled java 64 bit and filebot 64 bit and it didn’t seem to work out either.

So I used a gif as background to do the changing pixel color. However, the gif library isn’t compatible with processing 3.0+ so I have to use an image string to display frame by frame.

This is the error I got.

【Kinetic Interfaces】Week 3 Assingment Buttons

The effect I’m trying to achieve there is when the mouse slice over one bubble, play the bell. If slice two bubble at a time, play the drum. If the mouse moves vertically up and slices more than 3 bubbles, play the flute. However, I didn’t figure out a way to do the slice two at once coding so that I replaced it with hitting two at a time. The vertical slicing now is replaced with moving fast upward.

Game Analysis: This War of Mine (Krom)

“this war of mine”的图片搜索结果

This War of Mine is a real-time strategic/ decision-making game. The game was inspired by Sarajevo during the war of Bosnia and Herzegovina. The objective of the player is to survive the war with limited resources and extreme climate. The game ends when the war is over or all player-controlled characters died. All the resources such as food, water, and other materials are shared by both player and none player-controlled characters. Therefore, the main twist in this game is the survival between the player and the NPCs. To what extent will hunger and fear trigger the moral trigger and drive one to become a brutal person? More importantly, will your conscience allow you to stand with all that you have done? To me, these are the important the question that this game pointed out to us

At the beginning of each play, there will be a group of characters (usually 3-4) for the player to start with. There will also be some half NPCs that will ask the player for shelter during the play. Last but not the least, the pure NPCs. Some of them are just ordinary people that are trying to survive just like the player, some of them are mobs and soldiers trying to guard their own property and not willing to share no matter what happens. The resources in this game commonly appear as water, foods, woods, spare parts and materials that player can use to make tools. When first playing this game, there will be a tutorial level/day to show the player what is available to the player to collect and what are the needs to satisfy (hunger, thirsty, emotion). After that, the decisions are all up to the player.

“this war of mine”的图片搜索结果

Each character will have their own moral ethics. For example, some characters are more likely to feel sad about stealing from the NPCs or the death of their fellows. If the player doesn’t care about the ethical needs of the characters, they will commit suicide sooner or later. Take food as an example, the player can choose to scavenge cans in the ruins or build up traps and plant their own vegetables instead of stealing from the NPCs. As a result, the NPCs will live, and the characters won’t feel bad for anything. For some player, of course, they want to be nice from the start till the end. However, the decision becomes extra hard when there are fewer and fewer food and other recourses available. The number of the canned food is fixed for each map. Once the player consumed all of that, they need to look for more ways to make more food. As it is mentioned above, setting up traps to capture animals and cook it is a good approach. However, the player needs to build up a workstation and upgrade it before able to make traps. This requires far more materials and woods that the player can gather within the first few days. Therefore, the only choice for the player in the starting stage is to either to steal or starve. Followed are some classic action-outcome relationships that will occur in this game.

A. The player chose not to steal and try to become self-sufficient at the very begging of the game. They will devote all resources to setting up the workstation and making traps.

B. The player tries to steal food from tough people that are likely to survive without the food taken by the player. The downside of this decision is the tough people are well armed. Confrontation with them will result in injury.

C. The player tries to steal food from mild people that are not so likely to survive with those foods. The mild NPCs will beg the player not to be brutal on them. They will not harm the player.

If the player chose A. Their character will starve for a few days. This will lead to hunger and disease, which raise the need for medical supplies. Without the good luck, the player is not likely to make it until getting self-sufficient on food and water. If the player chose B on the long term, the characters will not starve but will get hurt. This needs bandages and medicines, which are also highly precious resources during the war. If the wounds are not treated properly, it will lead to the death of the character, which will result in depression of other fellows in the shelter as well. When it comes to C, the hunger problem is solved as well. However, this will lead to the death of the NPCs. When the player revisits the small house that they took food from after a few days, they will find out the old couple that begged you not to take their food are dead already. The characters will feel sick of their own acts and begin to hate themselves. They will become very low-spirited and are not willing to struggle to survive anymore.

We can see that it is always going to be very hard to survive through the war in this game, not to mention when winter comes, the water froze, and the temperature drops low, all your hope is a small radio that’s broadcasting the news of the day, wishing the war ends tomorrow. Not all of us get to experience such a hard life. This game makes people realize how fragile human is. The contrast between the intense game pace and the real-life shows the preciousness of our ordinary daily life. For me, the most important thing is that this game force player to think under what condition will we be willing to give up our moral ethics. This doesn’t only limit to the wartime, but other like students trying to get grade points by cheating, or bystanders when seeing other people in need.

“this war of mine events”的图片搜索结果