Week 11 Update

Due to focusing all my attention on social sciences this week, I haven’t made significant progress this week. I plan to create and testing some new layouts this weekend, per Rune’s advice and would be happy to post a second update based on these results.

I can write a bit about what I’ve heard from people on my most recent updates, first that I should focus making all the text legible on the persons face. This way they won’t have to go out of their way to read the text on the screen. Other advice I’ve received has been to try to improve the movement of the elements on the screen to make them seem more natural. An example I was given was used give the moving elements some sort of elasticity or spring-like properties to make interactions more satisfying for the user.



Collaborators: Mars Zhao and Pearl Basinski

Tech: unreal engine, speech recognition plugin

Game Design Document 

Project Description:
Like the waxing and waning of the moon each month, humans perceive time in a cyclical fashion. In reality, life runs in linearly which fundamentally contradicting our perception of time and our journey through life.

Fullmoon is an introspective virtual reality experience inspired by the life and work of Paul Bowles. By travelling through different life stages, the user is lead on an existential journey encouraging them to re-examine their relationship their own autonomy.

Fullmoon uses cinematic techniques to create an immersive experience. In addition, the experience makes use of teleportation, room-scale specialization, and voice recognition for a fully immersive experience.




Below are screen grabs from the experience

Scene 1: Study

Scene 2: Full moon rise

Scene 3: Childhood afternoon

Scene 4: Are you lost? 

Week 8: Progress

For the past few weeks, I’ve been trying to develop the webcam mirror component of my project. I decided to use a facial recognition javascript library called CLM tracker. My first step was to figure out how to actually use and implement the library to suit my own needs. The main uses of the library seemed to be face detection, emotion detection, and face morphing.

I needed to be able to determine the positions of different points that the library was reading and overlay text on top of them. CLM tracker’s facial model identifies 71 points on a users face. I decided to try to track one specific point on the tip of the nose. After finding the coordinates of the position of this point, I was able to try to place text over the point. There was a problem that surfaced when this happened, the text was reversed, or ‘mirrored’ along with the rest of the video captured.

The direction I was recommended was to try facial tracking in p5. There was a p5 adaptation of the CLM tracker and I was familiar with p5. With what I had learned from using CLM tracker, I was able to identify and track points on the face, then overlay legible text on these points.

From there, I was able to do some user testing, just with the facial and text tracking and the content. After getting these parts of the post completed, I thought I tried to determine the user interface for my project. I created some different layouts in photoshop in an attempt to determine the best way for me to design the installation. I’ve written more about user testing in another blog post.

After using the p5 version of the facial tracking, I decided I was a bit unhappy with the way that p5 was utilizing the CLM tracker. It lagged frequently, almost to a point where the face didn’t seem to be tracking. I revisited the original javascript library to try to make the text read in a more legible direction. After making some adjustments with the CSS in and with the position of the x-value, the text was able to be reversed. I replaced the text with an ellipse, which is what would place over this part of the face.

Now that I know text will be legible, I will move forward with adding the interface and text components.

User Testing

I divided the user testing into two different sections, the first was to test the interaction and the second was to test the aesthetics of the project. Users 1 and 2 provided feedback on the interaction. Even when I was testing the text overlay and reflection by myself, it was obvious I would need to find a different way to overlay the text. Most of the posts were much too long to be simply read.

User 2 remarked that the text seemed like too much to read. When I showed them the text in another form, they revoked that their initial assertion. I decided that it was important to convey the message in way that encouraged the viewers to read other people’s stories. If they felt that there was too much for them to read, then that would get in the way of the interaction. The user did say that they found the content relatable, they could relate the insecurities or unhappiness about their bodies.

When reading the different comments about certain facial features, User 1 said they could discern a theme throughout the posts provided, but it was not obvious that based on appearances that the block of text was related to a specific body part. User 2 echoed this sentiment.

I decided to ask the users how what they thought about potential layouts. I offered them different suggestions and they responded to them with their own ideas. Based on their feedback as well as my own ideas about ways the information could be conveyed, I made a few different layouts and asked users what they thought about them.

User 3 liked layout #2, which had floating boxes over the an entire screen that displayed the camera’s vision. They said they thought it would be more effective to have the whole screen ‘reflect’ the viewer, with text overlaying the viewer’s face. As for different floating diagrams and points, they liked the ones with blue ‘shadows,’ or the boxes on the left side of my layout.

Conversely, User 4 liked the first layout, which had the face in the middle of a yellow frame. They really liked the vibrancy that the layout provided, likening it to Buzzfeed or Vice. They thought the frame would work better if the project was web-based, as the facial recognition library would have a difficult time recognizing faces with a cluttered background. I would have no control over a user’s background in their own space. User 3 agreed with User 4 in regards to the immersive quality that the second layout provided if I were able to control the background. They liked the bottom left-hand box and the blue dot with a black outline.

Moving forward, I will probably progress with the second layout, but consider the first layout if developing this project for the web. I’m concerned with the lag that p5 gives the project and may re-visit the original library to try to see if I can make the text read the correct way.

Sound Scape

For this experience, we don’t want a lot of sound obvious sound. The sound design will be very minimalist because the piece is pensive and introspectional, so we’d like to create this sort of feeling with the sound design.

For each of the scenes, there will probably be room tone. In the study, there may be some sound filtering in from outside. When the user returns to the study, if we have the time we would like for them to be able to select records to listen to.

The desert scene will have the faint sound of wind through the sand in the desert. So far, we’ve listened to a few recording on YouTube.

The playground scene will also have the faint sound of children playing in the distance.

We will also be working on user interaction through voice commands, so we will need to have a voice asking if the user is lost. In this case, we can use the clip from the film The Sheltering Sky or we could find an actor to record for us.

Assignment 4: Concept Sketches, Target Specifications and Patent Review

Concept Generation

For this assignment, I’m building off of my proposal, which was an updated, more user-friendly air pollution mask. The approach I took to this design was one that Christian recommended, if it’s a problem for you, at least you have one potential user: yourself. Every time I wear an air pollution mask I think about how much I wish there was something better for me to use. Because I haven’t seen very many improvements on the product, I thought that I was the only one that cared enough about the issues that arise when wearing an air pollution mask.

One day in class we were discussion air pollution and potential solutions. A classmate remarked: aren’t the masks good enough? I countered by saying that there was a lot of room for improvement and another backed me up, specifically in regard to the issue of them rubbing off makeup. This exchange made me realise that the issues weren’t just my own issues, they must have been problems that everyone faced. To develop this product, I thought about all the issues I had when wearing a mask:

  1. Not aesthetically appealing (I kind of want to look like a Japanese delinquent teenager)
  2. Strenuous exercise (i.e. biking causes glasses to fog up)
  3. Makeup rubs off when wearing
  4. Single/few use

Alternative Concepts

  • A mask that covers the ugly mask

This mask would be something like a surgical mask, but with an interesting design and could cover an ugly, but effective mask. The patterns could be streetwear inspired.

  • Ugly mask

This mask focuses on the practical usages for the mask, less fog when exercise, reusable, and doesn’t rub off as much makeup. This is accomplished by switching the material of which the mask is made. I’ve heard that a cotton handkerchief can be used to blot makeup without taking it off, so this is where I would start with the material to use. Less fog escaping the mask would have to be solved by the material of the mask or by modifying the vents on the actual mask.

  • Sleek mask

My ideal, Japanese inspired mask, right material, minimalist design, replaceable filter, and embroidery on the design. This would also be streetwear-inspired, and probably look similar to the Nikka mask that is in my presentation to create a proper seal around the nose and mouth area. This mask will probably not be able to reduce the breath escaping and fogging up glasses.

  • Scarf

The scarf is a departure from the mask idea for the most part. The air filtering qualities of the mask would probably be reduced, but in its place would be less steam escaping and a much more fashionable mask. It would use replaceable filters and a fan to help circulate air.

  • Bike with filter

This idea comes from the designer Daan Roosegaarde and his partnership with ofo. A bike is fitted with a filter that filters polluted air, then blows clean air towards the rider. There is nothing more known about this project other than a diagram they created about the flow of air, so it seems they’re still figuring it out. Like the scarf, this filter cannot filter as much air as a mask would, but is functional for exercise, doesn’t rub off makeup, and is not an unattractive article of clothing. It’s also reusable.

Project Proposal

For my project proposal, I wanted to work on an updated air pollution mask. After living in China for a few years, health has become increasingly important to me, especially after exposure to air pollution. I wanted to deal with air pollution maks because I feel that the current masks do the job, but are not well designed beyond the basic purpose of filtering the air breathed.

There are quite a few opportunities for updating air pollution masks. The first and most obvious one being they are not aesthetically appealing or fashionable. I’m referring to the classic 3M mask for the most part, but many other masks attempt to be fashionable by creating ‘fun’ patterns, but really just look like menstrual pads. Even based on the initial 3M mask design, a much sleeker and minimalist mask can be created.

Another area to improve air pollution masks can improve upon is sustainability. It’s important to ensure the individual health and each wearer admits there is an environmental problem at play by wearing the mask. The masks solve the environmental problem of pollution for each individual that wears the mask, however, their production impacts the environment negatively because they are single or few use items.  Designing masks whose negative contribution toward the environment are reduced is another opportunity for improving air pollution maks to be explored.

Lastly, there are a few opportunities relating to the wear and comfort of the mask. First, if a user wears glasses, exercise such as riding a bike will make the glasses fog when the user wears a pollution mask. In addition, masks rub off makeup that a user may wear, so additional time must be factored into a commute if the user will wear a mask. These are both problems that can be reduced or solved with more attention paid to the design of the mask.

Due to the increase in air pollution awareness and increasing living standards of those who live in high air polluting countries, the air pollution mask market is steadily growing. The information about the market listed on the slide above can be found on DaXue Consulting and TechSci websites.

There are quite a few competitors over the pollution mask market ranging from single to multiple use masks. The single-use masks are relatively inexpensive and the multiple use masks (which are suggested to be replaced every 3 – 6 months) are priced accordingly. Note that most of these masks are not very aesthetically appealing. The Pitta mask looks the best of all of them, however only filters pollen and other large particles.

There is much room for improvement upon the current air pollution mask design and it seems that the perceived market need for an aesthetically appealing, comfort/usage aware, multi-use mask has yet to be met.

After presenting my idea for an update air pollution mask to the class, I left with one main improvement idea that I hadn’t considered before. Rather than creating a well-designed mask, a classmate suggested something like a scarf with a filter sewn into it could be effective and fashionable. Another classmate brought up the XiaoMi air pollution mask, which had a fan inside that helped cycle and filters air more effectively. The class thought it may be interesting to try to implement both of these ideas in one item. I’m interested in this proposal and would like to pursue it further.

Design Exercise

The gate to get in and out the of the school always confuses me. I want to use my ID card to get out and often stand in front of the gate with the card over the sensor until I remember I don’t need to scan the card to walk out. I’ve seen similar gates where you must scan to walk out and others where you don’t need to scan. The difference between our school’s gates and those that I see in other places is that the gates that don’t require you to scan out do not have a sensor. To make exiting the building more intuitive, they should require users to scan out.

Game Play: 旅かえる

I chose to play the game 旅かえる (Tabikaeru or Frog Traveler.) I’ve heard it’s been really popular recently, especially in China, where it’s called 旅行青蛙. Tabikaeru was created by the company Hitpoint, who developed Neko Atsume. Initially, I thought that it’s art style makes it immersive. It’s detailed, in neutral pastels and childlike, which conveys a sense of wonder. The music in the game adds to the childlike fun that the game conveys, which creates a very pleasing aesthetic that is nice to return to.


The game developers create a sense of immersion by giving the player ownership in the game. You name the frog, mine is currently called 小娃娃 after being named after myself because I didn’t know if I should enter my name or was choosing the name of the fog! In addition, you pick clovers in the yard as the game’s currency. When you’ve collected at least 10 clovers, buy snacks and other items, which you can then pack in your frog’s bag to prepare them for their next trip.

Immediately, I wanted to pick the items that I thought would be best for my frog, high in protein and vegetables, fewer sugars and grains. At first, I could only buy a scone, which wouldn’t be my first pick.


This leads my to one of the last aspects of immersion in the game: waiting. Because your frog it travelling, not you, you don’t have to be active on the app at all times. It may be better to leave you phone alone for stretches of time. Your frog will determine when they wants to travel and send back pictures and souvenirs from their travels. If you want to check back more often, some new clovers may have grown that you can harvest to get ready for your frog’s next adventure.



When I read the description for the class, I was intrigued. It seemed like it would help me answer some questions I’ve been thinking about for the present and future. For my entire life, China has been the manufacturer for the world. Recently, I’ve seen a how my parents’ generation has benefited from China’s economic growth. The summer in between Freshman and sophomore year, I interned at a nail polish start up that manufactured in China a product marketed to the US and Europe. The company has taken off and the owners have relocated from Shanghai to Barcelona. This summer I met a person who referred to China as a “gold rush.”

Having lived here consistently for 3.5 years, but having seen it grow along with along with me for the past 18, I’m not so sure I agreed with that person’s claim. I think maybe at some point, it was but now, I’m not so sure. For one, manufacturing is not as cheap as it once was. However, I want a more clear look into developing products in China and seeing what I could learn about China and what could apply to other countries that will take over China’s role as manufacturer. As I mentioned during the first class, I see this class as a practical application of another class that I’m taking, “Investing in Developing Countries.”

As for things I want to make, I’m not too sure. I’m interested in making an obnoxious phone case, in the same line of ‘decoden,’ but also with lights or something like that.

Another idea, would be a streetwear pollution mask. This would require a lot of research and development and may be above my skill set.