Final Writing Assignment| Capstone Studio | Dragonfruits

Once an expat woman comes to a new city, the labeling in the local community is inevitable.  In Shanghai, the urban cosmopolitan, stories may never be heard in the noise of labels. Most common labels that are encountered are an English teacher, an entrepreneur, a student, a traveler, a partygirl,  a company wife and others. However, lifestory is beyond labels. Lifestory is bigger than stereotypes. Lifestory is personal. The project Dragonfruits: Audio Stories from Shanghai expat women is a discovery of its residents. It’s the audio stories told by women who have a strong connection to the metropolis.

Several women were interviewed for Dragonfruits. However, despite willing to share their stories with the public, a few wished to remain nameless. In our society, it seems that the only inspiring figures the public is meant to see are men. Overly patriarchal society is on the way to change but women are still scared to share their success. As Chimamanda Ngozi Adichie once said, “We teach girls to shrink themselves, to make themselves smaller. We say to girls, you can have ambition, but not too much.” So many girls around the world are shrinking themselves. So many girls around the world are eating up their ambitions. So many girls around the world are uninspired. Dragonfruits wants to acknowledge the brave hearts who had no fear of making a decision to move abroad and start their lives over. Dragonfruits wants to inspire to be brave.

The web-based mobile app provides an opportunity for those who want to talk and those who want to listen. In addition to that, Dragonfruits’ connection to Shanghai is a direct proposal for city’s exploration. The stories are only available in specific locations of Shanghai where the women were interviewed. To be exact, the audio stories locations pop up at a distance closest to the listener. To ensure gradual familiarity with the city and smooth transition, Dragonfruits locations are secret. The listener may only uncover them by traveling around. As a result, exploration is unavoidable. As a result, another story is heard.


Dragonfruits Demo with the user’s location in Jing’an


DRAGONFRUITS SLIDES + audio snippet that was presented



Capstone – ProjectAR by Bruce

The year of 2018 marks the dawn of mobile Augmented Reality (AR). With new technologies like ARKit and ARCore being widely available to the developers and users, our phones can see the world like never before. Phones know where they are, and they track themselves spatially. This allows the user to interact with objects in the virtual environment through the screen as if they exist in the real world. It opens up new possibilities for interactive experiences. The problem, however, is that the screen has a limited size. In virtual reality, users can turn their heads immediately to see interactions happening around them, which is not possible in mobile AR. ProjectAR seeks improvements in this direction.

The experimental interactive game “ProjectAR” aims to explore extended immersion with the help of projection mapping. The player sees a virtual world through the phone working similar to magnifying glass and need to pay attention to the floor which is projected using a projector installed on the ceiling to look for targets to interact with. The game itself is simple: the player needs to find as many sprites as possible in a pond of fog. The little sprites inside the screen won’t move since they are trying to hide from the player. But the sprites may move when outside of the screen, and they will leave a trace on the floor. After figuring out where the sprites are, the player can blow into the phone to dispel the fog away and reveal the sprites underneath, and click to collect them.

ProjectAR aims to experiment and explore what works and what doesn’t with mobile AR and tackle some of its weaknesses using existing technologies. The initial idea for this combined platform was to build a spatial Twitter visualization, however, during user testing, it does not trigger interactions that much. Games are going to put the test of interactions and tracking capabilities to the extreme and encourage participation.

ProjectAR can be set up as an interactive installation using one ARCore enabled mobile phone, one computer, and one projector.

Project video:

Presentation Slides and project files:

WINDing Machine. Final Documentation. Ada


Slides link:


To NYC’s Museum of Modern Art

WINDing Machine is a wind vending machine that generates a wind pattern with real-time data using water and paint. Given the pattern in the water, audience can use the paper marbling technique to save the pattern on a piece of paper as a copy of wind.

Considering the amount of time nowadays we spend inside, seeing becomes the major way we perceive wind. In most cases, people can’t see the movement of wind but only the effects wind makes on other objects like trees and flags. WINDing Machine tries to extract the cause of wind, air pressure differential, and re-creates the pure wind movement for audience to see.

As an installation in the form of a vending machine, WINDing Machine will be triggered by user inserting a coin. The main component of Winding Machine is a tank of water. According to the real-time wind data from the Internet, a certain amount of paint will be dropped into the water. Then, Winding Machine will first rotate to a certain angle, then absorb water from one side and pump water from the other side, simulating the real-time wind direction and speed. The paint in the water records the water flow and presents the final pattern for users to marble and save for a longer time.

Water was chosen to be the main material in this project not only because water and air are both fluid interface but also because of its uncertainty and randomness. In the early stage research, a number of projects related to wind have been done from a scientific perspective with wind data precisely mapped to a visual form. However, in real world, the wind is never consistent or stable. If the weather report says the wind is at the speed of 5m/s, wind will never blow to you consistent at that speed. Using water as the main material, even if the raw data stays the same, WINDing Machine will still generate slightly different patterns. Thus, Winding Machine brings back the poetic beauty of wind movement relying on water as the interface.

Capstone_The Oracle_Jiayi Wang

In mythology and ancient time, the oracle was an enlightenment given from the divinities high above to us, the ordinary human living on the earth. It’s a revelation that will guide the actions of the receiver. However, in modern days, these words come not only from gods. With the development of the internet, we are given the opportunity to communicate with people from a far distance away. The anonymous identity empowered us to be anyone. Sometimes, it is another self that is hidden in the social life of the real world.

Oracle is a voice interactive statue that will respond to the audience in a very unsympathetic and intimidating manner. When the audience approaches the statue, the microphone captures every comment from the audience and send to the “god”, the chatting system, and thus begin a conversation between the audience and the “god”. The indifference and alienation that magnified on social media from an individual that is unreachable is the inspiration of Oracle. According to research, in online social life, most internet aggressive behavior is linked with the nature of anonymity online. The carefree online behavior code brings the users courage of saying anything they want to anyone without having to be responsible for it. For Oracle, the ruthless words from the stony statue are judgments casts upon the audience from the standpoint where the statue, the spirit behinds it, doesn’t need to be responsible for its words and it’s beyond the reach of the audience. Destroying the statue won’t stop it from talking nor put an end to the malice behinds it. A statue is eventually a statue, and the code is only ones and zeros.

Internet violence is not an easy problem to solve. The distance between users leads to the question of “why should I even care”? Therefore, it’s almost impossible to let all users online to behave themselves. More importantly, as the victim of internet violence, how should we act upon and take in the “oracle” from afar. The “god” chatting system is being mean because it is a piece of code hiding inside the computer, beyond the reach of the audience.

Capstone: Final Writing and Documentation – TA




CADMUS | Mission to Europa

Space exploration is becoming more and more automated with probes, satellites, and rovers flying forth from Earth’s orbit while humans haven’t gone farther than the International Space Station in decades.  This is hardly surprising considering how expensive and risky space exploration is. Yet there is still a call for human exploration, particularly with endeavors like Mars colonization. Something is still drawing humans out into space despite the greater dangers and higher costs.  Humans may not be the sole explorers of outer space anymore, maybe not even the main explorers, but their role is still important. Alongside the robots, drones, and AI that will undoubtedly come to characterize space exploration, human explorers continue to fill a critical role. This project aims to bring that importance, along with the abundance of technology that now accompanies space exploration, directly to the general public as an immersive experience.

This installation immerses the user in the sensation of space exploration, highlighting the technical and banal tasks that are involved in such an endeavor.  Users experience the process of donning a space suit and entering a strange and dangerous environment. However, part of their experience is to fix a malfunctioning probe that was sent before them.  The probe has three blinking LEDs, each with an associated port. The user interfaces with the probe by plugging a cable from their wrist controller into one of the ports. Then, by adjusting a potentiometer on the wrist controller, they can change the blinking frequency of the corresponding LED.  Once all three LEDs have been recalibrated to a specific frequency, a separate green LED on the probe lights up to indicate success and the user can exit and take off their suit. Although the user has been given this specific task to complete, their focus is not solely on the probe or themselves or any one part of the installation.  Instead they are surrounded by all of these elements intersecting to simulate modern space exploration. The individual components are not particularly sophisticated in themselves, but together they create a profound experience of the thrust of human pioneering for the next few centuries.

Capstone: Final documentation

Project description:

Since the invention of the first microscopes, scientists have been using magnification techniques to explore the internal workings of the human body. The development of sectioning techniques has allowed us to produce more realistic three-dimensional models, enabling scientists to gain a better understanding of biological structures.

These images, however, are rarely displayed outside of the scientific world and thus their aesthetic and educational value goes unnoticed by the general public. The purpose of the project was thus to create an immersive experience, through which the user could interact and reflect upon these images as a representation of a (part of a) human being.

Looking through a microscope and putting on a VR headset are similar in a way because they abstract the user from the outside world, from the dimensions of what is within sight and touch. And yet both technologies can be very introspective if used to look at the self. But how does one look at “the self” with a microscope? With an image? One of the ideas that I wanted to convey was that of looking into oneself through the eye of the machine.

By using VR as a medium, I wanted to emphasise the alternative “reality” that exists beyond what is visible to the naked eye, and thus that idea of the self as a multidimensional entity. Our bodies are the product of complex interactions between intricate hormones and communities of cells, while our minds exist within the realm of images and memories, in a constant state of edit and display. The intention was to explore the relationship between image and reality by exploring the human body in a hyperreal virtual environment.

Presentation link

IMA Capstone: Final Writing Assignment, Ariel

“Dining in Shanghai” is a tabletop projection mapping installation project featuring the content about local urban culture and the concept of “family” in China. The project is designed to provide users a dining experience where they can enjoy the Braised Pork Belly, a classical Shanghainese dish along with a story of a local family about the dish through the application of media and technology.

“Dining in Shanghai” elaborates dining experience with multi-sensory designs in sound, sight, smell, and taste. Users first watch a short animation that illustrates the family’s connection with the dish from three generation’s perspective. In the following part of gastronomy documentation about cooking with a traditional recipe, the vivid sonic environment of cooking Braised Pork Belly in the kitchen and the smell of the dish, which is triggered by heating, helps users to increase appetite before enjoying the signature home dish. Through the eating process, the user behavior data is collected to generate a customized output for individual diner with the different Chinese calligraphy of “食”.

The project aims to introduce food histories and family culture in China, but more importantly, it is able to remind users of their personal experience about food and family throughout the storytelling in the dining experience. The audience in different age groups are able to position themselves in the storyline where they have the strongest emotional resonance on real-life details and experiences, for example, the scene of kids playing games in lanes and the in the Chinese fine line animation.

The project is an experiment of using the psychologic effect on users as another element of the multi-sensory design, which involves users’ emotional interactions that help the diners look into the humanities and their urban consciousness behind media and technology. As the society steps into a digital age, this project would become an innovative example of integrating digital technology with humanities to make the precious oral history and urban development progresses accessible for people to explore or look back at in the future.

Presentation Slides:

Video Demo:

Visual Content:

final capstone writing assignment

Demo Video

These are my slides:


My project is a First Person Shooter(FPS) Virtual Reality Game.

The historical background is a battle between China and Japan happened in 1933, Hebei Province, China.

I wanted to improve the FPS gamers’ interaction with the computer,

I also want my users to think about the meaning of a war.

“better experience, more immersive, not very necessary, too expensive”

“The general idea was achieved. Things I have to improve: Interaction of the character(climbing up, the walking function of the animals, etc), VR technology itself isn’t advanced enough. More plot I could put in.”

“Professor Christian Grewell, Professor Chen Jian, Harrison Chen, Free 3D and Michel Yan.”

Final Writing Assignment:

Zhengyao Yang


To Moma

In Walter Benjamin’s art theories, observing art from a distance, and the unique approach to observe art gives this art piece “Aura”. That is to say, when one person travels to France and goes to Louvre Museum to see Mona Lisa, this particular way of approaching Mona Lisa gives it aura, and in this case, all the printed copies of Mona Lisa lose aura for not having this “distance” between them and the audiences. Art has to acquire the uniqueness to observe, and its distance with the audiences to obtain aura. There are many other ways he talked about how a reproduction can lose aura from the original piece, but we are only going to focus on these two reasons today.

Whether games could be art is still very debatable, but I think a VR game with beautiful interfaces and high immersive environment can make users feel the beauty of installation and human created art effects. When setting up a VR environment, the conditions of the room can impact the capacity of the headset greatly. For example, in the lab the some models I made look smaller and in my home they look bigger. I believe that in order to gain the best experience, I’l have to set all the environment by myself, and this makes my project irreproducible. I would say setting up the VR environment could be an analogue to an art curation, one player’s experience could be totally changed if the place changes, and this doesn’t happen to PC games, this is the unique experience my project brings.

Distance is also created by illuminating the reality and human created world. In my game, even though the users can see a human created world around them, and even touch, they can never feel the wind I installed, or smell the pine I “planted”. Their minds will constantly jump out and dive in. This creates a distance between the game and the user, it also helps my users to rethink about the historic background I embedded.