Kinetic Interfaces Final Project: Fish the Artists (Harrison & Oli)

Fish the Artists


Fish the Artists is a real-time visualization installation that captures the fish in a kinetic way and then generate artistic visual effects based on themselves. In this way, fish become the artists and all the equiments are installed around them.


After our midterm, a simulation system of a real pond, we tried to further develop our project around the key concept: fish. Inspired by a big show in Japan where thousands of goldfish are being exhibited in a dark hall, I tried to involve actual fish into this project. Since in that show, people are simply exposed to the original beauty of the fish. All the after-effects are the lighting, so I was wondering what if I could use some technologies from this class to add different splendid effects to the fish.

Continue reading

Game as Art Final Project: People Change (Harrison | Krom)

Ideation. Concept. Design.

As for now, if I directly say “People Change” comes from my first game project “Balloon”. So, let me explain 🙂

In Balloon version 0.2, the concept it tries to convey becomes a challenge to “an honorable way of living”. The purpose now is to trap the player in a difficulty situation where he can choose to pirate other balloons in danger for a small amount of resource or help them get out of the trouble for more resources since the other balloon won’t waste resource as fighting back. Whether or not the player could manage to save other balloons depend on how much certain resources the player has to help other. (For example, amount of water for fire; amount of amor for other enimies.)

In a word, I want the players to have some thoughts and feelings upon “an honorable way of living” by placing them into difficult situations where they need to make choices. So the feature of  “honorable” is judged in the choices they make. At very beginning, I insisted that the essence of “living” and this choice system in terms of how they are designed in my game was to help or not to help, to be alturistic or self-interest. However, as I go further and further, I found that their essence was carried out by the resource transmission. For example, giving gas, water, amor to others. On a conceptual level, all the action made by player is transmission of some kind of resources.

After this discovery, I stopped the work on hand and started the second round ideation. I picked out all the key words. Dilemma(“honorable way of living”)- Difficulty-Choice-Transmission. I moved on to think about the most efficient way to create a game that can let the player feel these 4 words at most.

  • Choice– In balloon, the design is often unreasonable because the choice system is built within 2 layers: to be alturistic or to be self-interest. I need to reduce it to 1. So I take out the alturistic part.
  • Transmission– Resources are a kind of private property. So I wondered what matters more than money if it is taken away from you. Then I got the answer: Identity.
  • Balloon—->Character- I made this decision for many reasons. 1)Since my game is more likely to be categorized as an action game, 3D environment is better for the player to feel the movement. 2)The balloon requires a enviroment setting of sky, sea and many other resources rather than ground and new 3D space of cartoonistic style. And players are more familiar with charater action game than a newly-introduced balloon based on my user testing 3)Taking a lesson from my midterm project, I moved the process of “whittling it down to a simpler version in order to get it done on time” to the beginning. So I tried to get the artwork settled as soon as possible. For the character, I don’t have to worried about it too much. They are all models downloaded from online, saving time from the balloon sprites. 4)I want to move into 3d development since the animator is a more particular function in unity compared to 2D sprites.


If I explain every script I wrote, I wouldn’t finish this documentation this week. So I decide to give a list of functions and features and explain some main functions like state machine.

Changing Color: SkinnedMesh,sharedMesh Renderer.material.color


Import all the packages, Use cubes to built a box. FX special effects for weather (originally for my balloons)


  • Camera Following: Vertor3.Lerp Angle Interpolation: Quaternion.LookRotation Quaternion.Slerp
  • CharacterController for movement: Input.GetAxis(“Horizontal”) & “Vertical” for x,y movement; CharacterController.SimpleMove(Vector3) represents direction and speed, unit:m/s
  • Animation State Machine: loop animation & one-time animation: Stand,Walk,Attack,Damage
  • Connection of animation to state machine: virtual attack button–
    EventDelegate normalClickEvent = new EventDelegate(this, “OnNormalAttackButtonClick”);
    GameObject.Find(“NormalAttack”).GetComponent<UIButton>().onClick.Add(normalClickEvent); Trigger animator.SetTrigger(“AttackB”); Bool animator.SetBool(“Walk”, false);

Other Soldiers: quite the same except for the AI function(automatic attack and findPath);

A for loop go through every soldier and pick up the closest soldier with different color.

HP and Damage system:Public interface ATKandDamage Class which can be applied to both player and other soldiers.

Soldier Spawn Script.

GUI: For the GUI part, I use NGUI 3.11 package and build each button with that. They can be easily connected to the script.


Models and Animation files – “Bingjingxingkong”-Baidu Netdisk

Unity3D video tutorials – Youtube

Scripts written by myself from scratch – Reference “Unity in Action: Multiplatform game development in C#”—Jiadong Chen


NGUI 3.11.4


Final Project Proposal (Krom): The Balloon | Harrison

Here is the link to my proposal:

In Balloon version 0.2, the concept it tries to convey becomes a challenge to “an honorouble way of living”. The purpose now is to trap the player in a difficulty situation where he can choose to pirate other balloons in danger for a small amount of resource or help them get out of the trouble for more resources since the other balloon won’t waste resource as fighting back. Whether or not the player could manage to save other balloons depend on how much certain resources the player has to help other. (For example, amount of water for fire; amount of amor for other enimies.

My initial proposal for this midterm project was to depict the everyday hardwork of a balloon rescue team, with a skill set and a choice system. Basically, my idea was to let the player encounter balloons in different kind of danger such as fire, lack of gas and so on. So the player had to weigh each balloon’s urgency and select the right solution to save them.

What’s different now is that there is no more an identity setting as rescue team, the player is just another normal balloon trying to survive on the sea. And the biggest challenge for this final project is to carry out the skill system which I don’t have enough time to realize for Midterm. Also, instead of constantly helping others without any reason, the logic behind the choice system now sounds more reasonable. That’s what skill set and choice system basically meant in my initial idea.

The previous design for how to end the game is that as the player could hardly maintain floating and falls into the sea, which symbolizes the cumulated negative effect on the rescue team and their spirits of sacrificing. However, in order to let the player have a stronger feeling toward his “disgraceful way of living”. There will also be a corresponding disaster similar to other balloon’s. In this way, the player gets into the shoes of previous balloons he saved or destroyed, waiting for help desperately. Whether the game will end or not depends on if another balloon decides to help the player or destroy him. ( a sense of reincarnation)

Things I need to refine

  • The flying experience (reduce the difficulty of maintaining flying)
  • Skill set and a key to call it
  • Sprite animation
  • A system to count the resources
  • Add an arrow pointing toward other balloons in danger
  • Replace physics falling down with timing function

Assignment 3: Pushing against boundaries (Krom) | Harrison

The 2 Sol Lewitt’s game rules I created:

1.Use a marker to decorate an egg.

2.Place this egg where its emotion is revealed.

Before examining on whether my creation is pushing against boundaries, I need to know what is exactly the boundary between game and art. One obvious difference between art and games is that you can win a game. As I got to know in the first stage of this class, a game has rules, points, objectives, and outcomes. Roger Ebert claims:” Someone might cite an immersive game without points or rules, but I would say then it ceases to be a game and becomes a representation of a story, a novel, a play, dance, a film. Those are things you cannot win; you can only experience them.” The answer to this questions leads to an obvious fact: the boundary between art and game is caused by the difference between win and experience.

Based on that, the provocations I try to create lie in the argument that one cannot prevent any experience from taking place if he wants to realize one objective. One can neither deny the fact that experiencing itself is an objective. The gameart that we try to realize is again the stage where these two sides decide to reconcile and to allow the existence of some specific creations. As a reaction, this specific creation strengthens my provocation from which I try to prove the possibility of a balance between win and experience.


The first rule of my game provides the player with artistic intentions. As I use the term “decorate”, players will intentionally step on the way towards beauty, or at least, not the opposite way towards beauty. Hence, to draw a beautiful egg becomes their win conditions. As Sharp posed the question “what happens if a game maker wants to create a game with artistic intentions? What happens when a game maker uses the language and idiom of games and their play as a medium for expression?” The answer to this question then introduces the experiences of art within a game.

Simply being immersed in a nice-looking environment, or building up a tower defend system does not in itself qualify a game as art. There are plenty of games that never intend to be art. In fact, most games are made with the sole purpose of creating income. However in my game, the current rule poses the intention to be art on the player but also gives them plenty of space to create. It tries to imbue the players with experiences that equals paintings, sculptures, songs but also makes the player feel free just as in a game where they have the right to make choices.

Continued from beauty in the art world, emotions and imagination are revealed in the second rule which bring art to life. While a complete definition of art cannot be provided here, a search on Wikipedia suggests that art is “the expression or application of human creative skill and imagination, typically in a visual form such as painting or sculpture, producing works to be appreciated primarily for their beauty or emotional power”. It is human imagination and the emotions that the players give to the egg makes this game art but also make this art piece a game if the mark and the idea is qualified to be art. Since It is important to realize that not all artistic intention meet the criteria of what is beauty. For instance, there is a very real divide between designers and artists which at its basic level is craft versus art. Where all games are designed, not all design is art. A craft is not necessarily a piece of art.

In addition to all the analysis above, I think besides making it possible to win and experience at the same time, this game indicates the unlimited development space for personalizing the artistic experience while still retaining the authority of the artist. In video games we find three distinct voices: the creator, the game, and the player. Those who play a game are following the story of the author and are bound by the constructs of the rules—but based on the choices they make. The experience can be completely personal. As in this egg game, one can use his piece of egg show to convey certain feeling or argument. If one observe the work of another and find in it personal connection, then art has been achieved.


Kinetic Interfaces: Week 10 Point Cloud & GUI (Moon) | Harrison

Since I have been abroad during the break, I decided not to bring Kinect with me in case of any unexpected damage. I finished this code after I came back. For this week’s assignment, I reviewed what I’ve learned from last class annd added acceleration to the particles. Besides, I linked the pointsize back to the depthdata just as what we did to the depthImage’s R and B value. I was going to add more osclliation effect to the point but then I discoverd the limitation of 3000 is really annoying because long term effects is not okay and will cause the framerate dropped a lot. So I stopped at the acceleration of the particle. For the GUI part, I also set the pointsize bar. So the effect of distance to the pointsize is then reduced by the number set by the user.

Kinetic Interfaces Midterm (Moon)- PondtoPan | Harrison & Oli

Pond to Pan

An interactive simulation of a real fish pond


Pond to Pan realizes the idea of player simply interact with a simulated fish hand with his hands. Thanks to Leapmotion, player can drop fish food into pond and cause a ripple simply by pinching his fingers or grabbing his hand. After growing a fish to certain sizes, player has to fish it and cut it into pieces on a cutting board.


After getting familiar with Leapmotion in week 6’s classes, I tried to think about what effect is more interesting to play around with one’s bare hands and also has a great visualization. I wandered around and discovered many cool effects of ripples. So I decided to realize the effect of causing ripples with fingers. But it still seemed too empty if I only do a simulation of ripples as a complete project. However, when I took the OOP we learned in class into consideration, the idea of making a fish class naturally jumped into my mind. What’s more interesting is that there could be a multilateral relationship between ripples and fish. Literally, fish can be attracted by the ripple. In order to let the player have more fun, we continued our conceptualization by adding a fishfood class for the player to play with.


Down below is the link to our conceptualization slides:


Process(3 main stages we went through):

Draw the fish and make it wriggle:

In order to simulate a fish in the real world, we are not satisfied with simply a circle. We did a lot research on the possible ways to do that. There are so many ways to add a dragging effects as a fish tail. But as we discovered Bezier Curve, we decide to stick on that. Another important aspect of bezier curve is that its data can be manipulated when we are coding the swimming movement of a single fish. Down below is the code we use to draw a fish and the code to make it swim.

Fish School Behavior (Flock Behavior):

To realize our most difficult part in this project, we used code by Diana Lange and built new functions of our own to make it more real. — move towards the same direction as the flock within sight– move towards the center of the flock within sight– moves away from the flock when it’s too crowded– moves away quickly when it’s too close. Down below is the code for alignment, cohesion, collision avoidance and separation.

Fishfood & Ripples:

For these 2 parts, they have the same characteristic of attracting fish and displaying themselves. So we decided to make them different by making the ripple’s power much stronger which means the ripple can attract fish more easily while food only attract the fish in a small distance. This design makes more sense or the player won’t have to cause ripple to lead the fish to the food.


During our concept presentation, Moon instructed us to built the whole project under a specific thesis. So we decided to focus on the imbalance between growing animals and finishing them. A lot of people in the world only eat meat or fish because they don’t see the brutal process that animals go through to become food for us to eat. I think we did it properly on setting the comparison, but based on the feedback, we lack coherence on the style in each stage. That’s our main problem.

Future Development:

In the future we want to focus more on the growing and nurturing side of the project. Professor Wuwei and Ceci provided us with the advice from different aspects. We are both inspired by the project “Aquarium” of Ceci which she simulated fish in a real water tank. We  want to integrate projection mapping and the kinetic. Adding on to our thesis, we can possibly project the whole pond on the ground and detect the body of the player and make player the food of those fish……


The code by Diana Lange:

The Balloon – Midterm | Harrison (Krom)

Here is the link to my proposal:

My initial proposal for this midterm project was to depict the everyday hardwork of a balloon rescue team, with a skill set and a choice system. Basically, my idea was to let the player encounter balloons in different kind of danger such as fire, lack of gas and so on. So the player has to weigh each balloon’s urgency and select the right solution to save them. That’s what skill set and choice system basically meant in my initial idea. At the end of the game, the player could hardly maintain floating and falls into the sea, which symbolizes the cumulated negative effect on the rescue team and their spirits of sacrificing. The main obstacle during my production process was to write the condtion event scripts that are not already in the playground project. So in order to be realistic, my simplified version of the balloon game is oriented at the collision.

In the game, if the rescue balloon collides with any other balloon, it will shrinks and its gravity will go up which shows that he had contributed some amount of gas to the other balloon, hence it is more difficult for him to float. The movement is realized in 2 parts. One is spacebar, which is responsible for the power. The other is arrowkeys which control the direction by rotate the balloon to a certain angle. I thought this way, compare to the method of only using arrow keys with a drag force, is more proper to stress the difficulty and professional flying skills of the rescue team. I am quite satisfied with the outcome except for I cannot write the script that limits its dip angle.

In regard to the other balloons, I only create a single prefab to keep the minimalistic and simplified style of my game. It would definitely be more interesting if there were many other kinds of balloons that are in danger. However, my balloon checks the collision and then it gravityscale goes down to show that it has received the help from the rescue team.

The meaning I want to convey is the spirit of self-sacrifice among the rescue team.One of the problem in my game is that in the first version, it is extremely hard for the player who is at his first time playing this game. So it results in a too short duration of gameplay, hence the struggle and the cumulated negative effects cannot be sensed by the player.

Down below is a demo of the game:

Links to the game:

Kinetic Interfaces: Leap Motion (Harrison)

For this week’s assignment, I focus on the pinch gesture of LeapMotion. I programmed a pendulum which the center of it is set everytime when the fingers are pinched. The center location is of course the location where the fingers pinch. I also worked out a shadow effect of the pendulum which adds a lot beauty to the interaction. I am also very satisfied with the sensitivity of the program. What’s more, for the ball class, I also detect its collision with sides and bounce it every time when it hit the edge. And the color is always changing like a rainbow.


Analysis of an Artgame: Limbo (Krom)

“Limbo” is a puzzle-platform video game developed by independent studio Playdead. The game has influenced the community by its excellence since it was released in July 2010 on Xbox Live, and has since been ported to several other systems, including the PlayStation and Steam. In my opinion, art must stand the test of time and that proves Limbo’s characteristic of art from the very beginning. Limbo is a 2D side-scroller, incorporating a physics system that governs environmental objects and the player character. The player guides an unnamed boy through dangerous and cruel environments and traps as he searches for his sister. His sister died so he came to pursue her into the limbo. “Limbo” is similar to the movie “Inception” in the way of how delicate their intensions are. Even if the player completes the whole adventure story, not everyone can get through what the game is about, or in other word, what meaning “Limbo” tries to convey. However, the thrilling experience of the adventure and the meticulous design of puzzles will definitely leave a deep impression to everyone. Except the fact that it can only be played once because it is an adventure game, it is perfect.

The game only requires two additional controls—jumping and grabbing—outside of the normal left-and-right movement controls, to keep the game easy to play which means the boy has no special abilities at all. The minimalistic style separates Limbo from many other adventure games since the main character usually has some special skills for the player to have fun with in other games. Under the circumstances that everything is quite simple in Limbo, the player has to seize every detail and pick up on subtle clues from the environment which usually costs a huge amount of dead trials. Although the game grows out of the original “Super Mario” game which is under the genre of persecution, Limbo still packages itself as a gothic blockbuster with a bunch of good-looking levels when the player suffers from unexpected deaths all the time. However, those sudden deaths generate another highlight of the gameplay as player will not feel over-frustrated because deaths are blended into the game so delicately that they create an ingenious balance between frustration and curiosity.

Limbo invokes emotion in the player by making incredible use of an all-grey palette. People loved Limbo because of its dark atmosphere. The game is eerie, chilling, and just plain scary at times. It is a nightmare, albeit a beautiful one. The main characters bright eyes stand out as he wanders the dusty forest in the first half of the game. The grey background makes the world of Limbo mysterious and alien, exactly how it acts in a game design sense and the story that complements it. I absolutely love Limbo. I get lost in its haunting and ambiguous world. What really appeals to me is just how captivating it is, and how the player can perceive it in different ways. Following the minimalistic color setting, the absence of dialogue and many depressing soundtracks also add to the atmosphere of loneliness and horror. Such design enhances the terrifying impression of hell. However, it is not the fear to the hell based on traditional understanding, but the fear to a lifeless environment. Meanwhile, many levels convey the twist and variation of the plot without any sound which can be viewed as a good example of affective visual interaction.

I feel that what Limbo is trying to convey is the boy’s feeling on the moment he died. On that moment, he got through the limbo and reunited with her sister in the hell. Maybe this world of limbo lies between the real world and the hell as a tunnel connecting real world and hell. Several seconds in the real world can probably become hours in limbo (as it is shown in the movie “Inception”) just because people are so quick in mind at that time, thus it accounts for why feelings like fear would be visualized as some objects close to human nature instead of some detailed facts. An accident caused the infortune of the brother and sister. While the sister takes a step deeper to the hell first, the boy enters limbo. Of course, the sister has already reached the hell because the sister appears twice. Once at the very end, the other time is in one of the puzzles where everything is spinning which is very likely to be indicating a chaotic state in limbo. Inside this place, the boy’s feeling will be represented as objects: Spider—fear, hanging man—death, clamp—bloodiness, gears and factory—falling and forest—calmness. If the character wants to get to the hell, he needs to go through all of these.

In order to convey such delicate meaning, developers built the game’s puzzles expecting the player to fail before finding the correct solution. Playdead called the style of play “trial and death”, and used gruesome imagery for the boy’s deaths to steer the player from unworkable solutions. The finished game was to present no tutorial text to the player, requiring players to learn the game’s mechanics on their own. The game was purposely developed to avoid revealing details of its content; the only tagline the company provided was, “Uncertain of his sister’s fate, a boy enters Limbo.” This was chosen so that players could interpret the game’s meaning for themselves. Also, sound arrangements helped to evoke emotions. As I mentioned before, the acousmatic music was intended to leave room for the visuals as well as the interpretation by the player in the same manner as the game’s art and story. This helps with immersion within the game by making no attempt to control the emotional tone.

Kinect Interfaces: Pixel Manipulation (Harrison)

For this assignment, I build a particle system with which the user can see the water lilies created by the great painter Claude Monet in the effect of a fountain. The idea comes from the actual feeling that when I was looking at Monet’s work, I feel the tension of between the water lilies and the pond. I can feel the potential motion in the water so I decided to make it more dynamic with such effect. When the user click a point on the canvas, there will be a generating fountain welling the pixels of Monet works and as the user create more system, the whole painting will be revealed.