Week 6

Leap Motion & Projection Mapping

Class Overview

Lecture 1

  • Assignment review
  • Introduction to Leap Motion
  • Practice the button states (triggered, pressed and released) with Leap Motion
  • Gestures with Leap Motion

Lecture 2

  • Introduction to Projection Mapping
  • Basics of Syphon and MadMapper
  • Keystone library and Syphon library in Processing
  • In-class Exercise: Practice Projection Mapping onto multiple surfaces


References – Leap Motion


References – Projection Mapping


Other Projection Mapping Tools



  • Prepare Midterm Project Concept Presentation.
  • Create a simple interaction with Leap Motion, including the button states (triggered, pressed and released).
    • Utilize “abstract” button and button states, that were topics covered in week 3.
    • If you want to challenge yourself, integrate projection mapping into your leap motion sketch. For example, with your finger, you will be able to press a button projected on a table or trigger movements of particles on the surrounding area.
    • Please feel free to reach out to the instructor if you struggle with coming up with ideas.
  • The exercise should be uploaded to Kinetic Interfaces google drive folder by midnight every Saturday so that the instructor has enough time to review your work. You should upload not only your source code but also all of your media assets, such as image, sound and video so that the instructor is able to execute the program.
  • Additionally, your work process should be documented on the documentation blog. Your documentation needs to include sufficient amount of screenshots and video captures along with descriptions.




Leap Motion


  • Download Leap Motion App here!
  • And the processing library below.


Projection Mapping

For Mac

For Windows


_Tips for Projection Mapping

Fabric Material for Screen

  • White
  • Reflective (but not too much)
  • Mesh (The grid size can be bigger if the distance between the screen and the audience is further away)
  • Not elastic (should not be extended easily)

Steps of Projection Mapping Set Up:

  • Hardware (with projector):
      • As close as possible with minimal use of zoom
      • With slight overflow of screen for the desired area
      • This will allow for the brightest and clearest projection possible
    2. LENS SHIFT: Adjust position with Lens Shift (if the projector has this function)
    3. ZOOM: Adjust zoom in adequate amounts
    4. FOCUS: Adjust the focus until the projection becomes as sharp as possible.
    5. KEYSTONE: Adjust the projection area to be as straight as possible using Keystone
  • Software (for example, with MadMapper):
    1. GRID & GUIDELINES: Project a grid or guidelines
    2. QUAD ADJUSTMENT: Make fine adjustments to the 4 corners of the projection area utilizing the grid or guidelines
    3. EDGE BLENDING: Blend the edges (if creating one screen with multiple projections)
    4. Add more details (segments or curves)



What Is Projection Mapping?


ENVISION – Step into the sensory box(2010)

3D video mapping projection brings buildings alive(2009)

Shanghai Disneyland fireworks


Amazing Dance and Video Mapping Performance

Tracking + Projection Mapping



Most Insane Immersive Movie Experience EVER, Part 1

Most Insane Immersive Movie Experience EVER, Part 2


_ Surfaces
3D Mapping on a transparent Jaguar car Full HD

Face Projection Mapping

FLUIDIC – Sculpture in Motion – (kinetic sculpture) full documentation 2014

Holographic Performance?


_ Scale

pinwall | interactive game projection | augmented reality


_ Narrative
Roger Waters – The Wall