HATCH/宝贝 Final Doc Post (Saphya + Nicole)




Game Design Document_slide1

宝贝/hatch is an augmented reality dating game that encourages “real world” interaction.

Game Design Document_slide2

We realized that the problem with the culture of online dating is that people are less likely to meet in person, which is an issue we hope to rectify through our app.

Game Design Document_slide3

A combination of Neopets and Tinder, users will match with each other based on common interests in their profiles, thus enacting an event where they must find and foster a pet.

Game Design Document_slide4

These rules are put in place to establish a serious minded relationship between two people. We want users to use our app as a means for physical interaction, not to prolong virtual discourse.

Game Design Document_slide5

Users can obtain resources in different places within the academic building to raise their pet. In the cafeteria one can scan for food; in the cafe one can scan for treats; and at any water cooler one can scan for water. The bare necessities. Users are also urged to visit their pet together every now and then to give their pet a love boost.

Game Design Document_slide6 Game Design Document_slide7



Screenshot (451)

The foundation of our app is controlled by four scripts: Time Management, Heart System, Camera Button, and Tracking Event Handler.

Time Management controls the active states of the hearts in the GUI. Using an array, this script will set the active state of each heart in the array to false in thirty second intervals through communicating with a function in the HeartSystem script called HeartDeletion().


Camera Button is connected to the lower button in the GUI that activates the ARCamera. This script is attached to both the coffee button and the pizza button and takes several parameters associated with the button identity to pass into the Tracking Event Handler script. If OnTrackingFound() returns true for the specified image target, then HeartAddition() from the HeartSystem script will initialize.


Tracking Event Handler cross compares what is being scanned in the ARCamera with the image target the CameraButton script has sent it. This is to ensure that the camera doesn’t deinitialize after scanning just any image target in the database, and to make sure that the qilin receives the correct  icon popup in the game.


Heart System is connected to both the CameraButton and Time Management scripts, and controls the qilin’s animations and appearance. If the qilins hearts reach zero, the qilin will turn into a pile of bones, otherwise it will be a cute qilin.


Here are our markers:

fdpizza fdcoffee



We tried to incorporate multiplayer compatibility so that couples can chat and take care of the same qilin. We first did this through the Unity Multiplayer Networking tutorial, however it did not work between PC and mobile phone. This is because there was no server to connect the two devices. Next, we explored Photon Unity after getting advice from Sean. This was better, because Photon uses a cloud that can be accessed just as an API is used. In the short time that we had, we prioritized perfecting the Vuforia camera and the appearance of our app over the networking component. In the future, we hope to finish our work on that part of the app.


Baaria and Rewant – Quaternion Identity

Quaternion Identity

Baaria Chaudhary & Rewant Prakash
(baaria@nyu.edu, rewantprakash@nyu.edu)

Quaternion Identity is a location based, theatre-style augmented reality game experience. The game is set up in the academic building at NYU Shanghai and takes players on a journey to solve riddles and find clues to reach the next level. Players are given an iPhone with the game configured on it, which they then use to scan different markers and unravel different clues. At different stages of the game, players also interact with live actors to access clues and continue the experience. 

The theme for Quaternion Identity game is privacy and data security, inspired by a variety of different events that include fake news in US presidential elections, Ashley Madison data breach, ransomware attacks as well as the recent increase in cyber attacks.

Unity, Vuforia, IFTTT, OBS, C#, Adobe Premiere, Adobe Photoshop,

iPhone, Skanect, Green Screen,

Flashdrive, Book, Tea table + experience, 3 rooms with different experiences (/Performing stages)



HW 10 Sound, Daniela & Shawn

We use unity to make the 3D spacial sound.

Make sure the spatial blend is in 3D mode for all the audio assests.


Max distance can control the area of the max for audience to hear. Here we adjust to to make sure that the player can only hear what each character says when the player get close enough. We adjust it differently for the guard so that the player can tell how far he is from the guard by the sound of laser beam.


Here is the sample of effect.

Final Project Shanghai Storylines – AR

Shanghai Storylines

Team member: Tylor

Together with Tylor, we planned to create an experience that will allow the user to see how Pudong looked like 25 years ago. We wanted to be as realistic as possible in creating the experience of the old Pudong. The role of the player is to explore the old areas of Pudong that were not even close to the level of development we see today. We wanted to give a bit of nostalgic look to the project and be as realistic and precise as possible.

We have focused on the historiography element of the project. Looked at the old maps of Shanghai and the old pictures. Later compared our findings with the present day data. The next step was to recreate the world from the few decades before in the Unity.

Screen Shot 2017-05-24 at 02.21.08 Screen Shot 2017-05-24 at 02.20.30

The challenges were a working with Unity and Euphoria as I have never worked with unity before. Another challenge was collecting the historical information about Pudong as this part of Shanghai was not well documented. We have spent a lot of time watching tutorials on youtube learning how to make an accurate representation of the terrain and listen to the advice of Sean who helped a lot in the initial part of our project. I have done the beginning of the train in the unity and Tylor developed it and gave a final look to the project as well as worked with Euphoria.

We wanted to create a realistic transition for a user from the real world into the virtual world, therefore, we came with an idea to place our marker for a smartphone on the actual frame of the window. I have created a frame of a basic window together with the help of Marcela in the IMA lab.

We really liked the idea of the project that pushed us to discover the city in which we live in. And challenged us intellectually in the technological areas that I had no previous expertise. Without the project, we would have never know about the Lujiazui’s Church, a graveyard for the international sailor’s cigaret factory and it’s nightlife transformation.
Our project needs an improvement in creating the more immersive experience for the user. It could have been created more immersive realistic and overall more convincing. We believe that it could be much more realistic as the user like through the window into the old Shanghai and actually feel that it is transported back in time.

HW9 MR interview by David, Reine and Diana

We are inspired by the game Sims. So we created a conversation between a people who happened to meet a sim inside the house. I did the 3D scan by Skanect with Kyle’s help. And then Dave and Reine did the interview. Because there’s something wrong with rigging, the model looked kind of weird. And it felt really strange to see someone else who controlled the movement of “me”. But it’s interesting to see how to animate 3D models by Kinect. I can feel that there are many possibilities with technologies like this. Maybe we’ll try to create something more fun with it.



TS4_Logo_Plumbob.jpg MTS_stonee206-1429879-14


Shrine Guardians Documentation by David, Reine and Diana


Shrine Guardians is a mixed reality experience designed in Unity that was designed to tell the story of NYU Shanghai’s place in the city of Shanghai. NYU Shanghai is a mix of Western and Eastern cultures, and the experience was crafted to grab hold of some of the values that come from a multicultural university such as NYU Shanghai.

What we aimed to accomplish and why:

We wanted a cohesive and minimalist story that utilized mixed reality storytelling elements. We wanted to work on Unity and a consistent style that stuck to the original goal of the design document in order to keep our group on a straight and clear path. Our skills were limited, but we made sure to keep track of what we could and could not do in order to have realistic goals, and a working project by the end of the class.

What we did:

  1. We decided the main character for the whole story: A “chicken” grows into a phoenix with the help of the player. We created the chicken model and downloaded the phoenix model.
  2. We made markers some ink-style markers, and we also found and created 3d assets for the story, like a Mobike, Pipixia and other models.
  3. Code for the whole frame. We tried the touch function but it didn’t work on Android.Then we changed it to click function. Our players need to talk to the chicken first, then scan different markers and find the Pipixia. Then they need to go back to the chicken again.


How we did it:

  1. We decided to create Vuforia markers using traditional Chinese calligraphy and paper. Choosing this style contributed to the theme of the project – the NYUSH hybrid of Eastern and Western aspects. Assuming the game takes on an Easter-egg hunt style, the eggs themselves should themselves be engaging to the participant.
  2. Each asset placed into the project is commonly found in the daily life of NYUSH students, including cigarettes, a Mobike, bubble tea, noodles, and Pipixia (a popular Chinese meme).


a.) Cigarettes – author – “michael -mihalyfi” https://www.cgtrader.com/free-3d-models/household/other/cigarette-lit

b.) Bubble tea –

c.) Mobike – author – “ufukufuk” – https://www.cgtrader.com/free-3d-models/vehicle/bicycle/bmx-0f5375f9-8634-4450-bc3b-027b9b939824

d.) Noodles – author – “HQModel”


e.) Smog (pollution) – author – “JK REMO”


f.) Bamboo – author – “squidward” https://www.cgtrader.com/free-3d-models/architectural/decoration/bamboo-raft

g.) Pipixia – author – “edi”


h.) Phoenix – author – 3D学苑


i.) Dancing girl – author – “drachengeifer”



Links to project files:

  1. Unity Package of project(probably not the most current build): https://www.dropbox.com/s/t6qyu0efni7g0eq/shrine_guardians_unity_package.unitypackage?dl=0
  2. APK File of Project(probably not the most current build): https://www.dropbox.com/s/jbw3r81n4a7cd9g/shrine%20guardians.apk?dl=0
  3. Gist file of main C# script, (spaghetti code ahead, be warned, also probably not the most current build):https://gist.github.com/dave-santiano/2cea3e9434d03c80bccc7b37658202d7


Video of project working:



MR Interview, Homework 9 – AR

This assignment was a lot of fun. We have quickly developed a model of the Vladimir Putin and used it in our project thanks to helping of Prof. Christian. We used a Microsoft Xbox Kinect to detect the movement of one of us, green screen and knowledge of our professor to put everything together.

Tyler was the one conducting the interview and I was the one who by its movement manipulated the skeleton and the model of Vladimir Putin on it. We could have thought more about the script. I’ve learned about the new technology that needs improvement but shows a huge spectrum of possibilities in the future. Iw we would have done it next time I wish it could be more planned. I would like to control the movements better and give a more realistic feel to this exercise.

It was a very good experience that I hope I will be able to repeat in the future after finishing NYU Shanghai.

The Core Mechanic Documentation Homework 8 – AR

Title: Shanghai Storylines

Team member: Tyler

Presentation Slides:


Screen Shot 2017-05-23 at 16.38.34

Together with Tylor, we plan to create an experience that will allow the user to see how Pudong looked like 25 years ago.

The role of the player is to explore the old areas of Pudong that were not even close to the level of development we see today. we want to give a bit of nostalgic look to the project and be as realistic and precise as possible.

Screen Shot 2017-05-23 at 16.38.57The challenges are a modelling that would be realistic as we have never worked with unity before. Another, less technical challenge is collecting the historical information about Pudong as this part of Shanghai was not well documented

Screen Shot 2017-05-23 at 16.39.08We would love to use Hololens but unfortunately due to the increased accessibility and limitations in our technical knowledge may use the smartphones instead.



Pop The Bubble Final Documentation (Maggie & Ale)

Pop The Bubble by Maggie & Ale

pop the bubble

Our Project

Pop The Bubble is an augmented reality experience using Google Tango that lets you walk around a Chinese neighborhood. The aim of this app is to allow NYU Shanghai students to experience Shanghai beyond the school.

How to Play It

You will walk around the neighborhood and encounter different objects/people that will give you a better understanding of a typical Chinese neighborhood.

Binaural Recording

Even though we didn’t manage to get 3D scans of the people at Jinqiao park, we were successful at capturing the feel of the park and its people through binaural recordings. In our recordings, you can hear old people chatting, playing cards and even a little girl performing to traditional Chinese songs.

Working with older Chinese people was an interesting experience. We went out into Jinqiao park in order to get 3D body scans and binaural audio of people leisurely relaxing in the park. Upon approaching a group of men playing cards, they were all very excited to interact with us, and what we were doing. They played with the google tango, and touched the ears of our binaural microphone. But, once we began asking them to aid us in our project, they became much more closed in. It took only one man to say no for the rest of the men to agree with him. I think getting the “leader” of the group to say yes is the key to getting the rest of the people to say yes. Though, in order to get the first person to say yes, I think that we would have needed to build up a level of trust with them that we just did not have the time to do this time around. That would be the next steps in this process.

binaural recording 1binaural recording 2

binaural recording 2

Evolution of the Project

Hololens > Google Tango

Originally we wanted to do a hololens experience. But, while presenting our project progress with AJ, he urged us to take advantage of the tools we were using, and not just use it for the “cool” factor. Therefore, we switched to the Google Tango, because we could utilize the depth perception of the device to have a user walk through a neighborhood.

Multiple Communities to One

We started out with many many places we wanted to capture, but ended up focusing on one place that had many characteristics of all the places we wanted to portray in our experience. We wanted to show a lot of things about everyday Chinese life, and I think we did that well in the experience we created.

Future Steps

In the future, we would love to get full body scans of people to import into the scene. We would also like to do more photogrammetry of real things to import into the scene as well. As of right now, we did extensive searching for the 3D models we have in our scene now, in order to create the most authentic experience possible. Our project also included a lot of external research into how we should create our environment, by observing real life environments. We would love to get even more detailed in our scene by adding more aspects of everyday chinese life, like a lianjia umbrella before you walk in. A line of bikes alternating mobike and ofo, a street food skewer guy outside the front gate, etc.