I am a senior designer who loves to explore ideas, methods, and technologies that can be used to create new experiences that immerse users in their digital content.

I am a senior designer who loves to explore ideas, methods, and technologies that can be used to create new experiences that immerse users in their digital content.

Click Here to Open Project

Made to be a simple tech demonstration, this interactive uses an Oculus Rift and Leap Motion controller to build molecules and play with atoms.

My Role

For this experimental project I served as the UX designer, creating wire-frames and a generic graphic set.

Challenge

Utilize a Leap Motion controller to create an educational experience where students can build molecules by grabbing atoms.

Process

Leap Motion is a sensor that can detect your hand and finger position and orientation in real time. The initial assignment was to create a table top implementation with a large screen in front of the user, as seen in my wireframe on the left. So my first step was to become very familiar with the Leap by reading the documentation, playing around in their various applications and identifying what sort of detectable gestures we could utilize. Below is my wireframe of those gestures.

I set out trying to think how we might select atoms to build our molecules and my mind went to a wall of the periodic table. This ended up becoming too complicated to maneuver so it got scaled down to only a selection of atoms and upon instruction from the creative director they were turned into ping-pong balls with the atomic letters on them. From there I worked on how the work-space should be arranged and the types of information that should be displayed. My first wireframe pass is below.

Using this as the base I wireframed out the functionality of grabbing the atomic ping-pong balls and bringing them into the circle where they snap together to form the molecule.

After discussing with the developers and spending several days with the device and their latest Orion drivers, it became apparent that the direction Leap Motion was taking is towards virtual reality so we modified the project to exist in VR. This resulted in some slight wireframe changes like adding target circles, giving menus z-depth, and adding a floor for bouncing physics fun.

We also wanted to be able to view the molecule at different angles so I created the wireframe below that details how someone might grab and spin the object.

Final Product

Below you will find a gallery of the final wireframes that created for the project, and at the very top of this post you can watch a video of the final product. As the development progressed the team and I would discuss how things were working and evaluate the functionality. Along the way some features were modified for a more responsive experience, along with adding some fun things like all of the atoms bouncing away at the end.

Final Wireframe Gallery

Click Here to Open Project

The purpose of this project was to create an interactive attraction that would help advertise vendors in the marketplace as well as help people find those vendors on the show floor. I have explored various software from Flash/Starling to Html/Javascript libraries to Unity in the search for the best platform for these types of projects and what I discovered is that essentially, it depends on the use case. For the table project below I used Flash with Reactivision to achieve the touch and fiducial recongnition with graphical interactivity. Below is a video with Sabre (Kimberly Weigend) from the Fragdolls interviewing me at MAGFest 2013 about my table. It was a lot of fun to see everyone’s reactions and I hope to do something bigger and better in the future.

The benefit of the DIY versions of these types of projects is that the technology and plans are available on the internet for anyone to access and you can build one for a fraction of the cost of the professional products. Of course if you want better performance, you need to shell out more money. The table I built for MAGFest was the lowest budget I could manage(around $800), but still performed fairly well even though it had some bugs such as not recognizing small touch surface area at times and misinterpreting the exact orientation of the symbols on the cards. Those problems could all be solved with better equipment.

Other Videos:

A video I took of people using my table

For those looking to build their own:

The technical term for my table set up is Rear Diffused Illumination. I learned a lot about what I know from videos on youtube, the NUI Group forums, and bought some parts from the Peau Productions store(same guy from youtube). I bought low cost infrared light emitters that shine up through the table surface which is comprised of a rear projection material stretched over a home depot bought acrylic. Anything that touches the surface reflects that light back down into a Playstation Eye camera that has been modified to see only infrared light. The PS Eye sends images to a program called Reactivision which recognizes symbols on the back of cards and finger tip sized blobs, registering their number and orientation and sending that information to an Adobe Air program written in Actionscript 3 using the TUIO AS3 library. The Air program is then projected onto the surface rear projection material. The program I used at the show was running the standard Flash engine, which I have learned is rather clunky and since learned the Starling game engine for flash which purports to make use of full 3D hardware functionality allowing for greater performance on a variety of devices.

In Development video

Click Here to Open Project

Before this video was made I had a phone call with Cortina Productions and scheduled an interview. I wanted to build something to bring to the meeting but only had 5 days. The interview then got delayed another 3 days so I had a total of 8 to get it running the above video shows the end result. Spoiler alert, I got the job. The video below shows the inside of box and early progress.

Click Here to Open Project

This is a functioning concept for an interactive kiosk. This project spans not only the front end user experience but also the content management side of how to best get varied content from many sources. The video goes into detail at length, but the reason for this project was simply that I enjoy making interactives and I like the idea of providing a platform for market vendors to attract attention. At the same time it would give festival goers something entertaining and informative to play with.