Charles Yust, Unified Field Inc.

Interactive Software Engineering

What can others learn from the successes and failures of the way you’ve bridged the digital and physical realms?: 

In many of our projects, the physical and spatial arrangement are integral to the success of the digital portion of the project. The physical-digital relationship is such that we strive to include design technology considerations and personnel as early as possible in the process of building each one. This is important for dictating the technological and hardware requirements before the physical design is completed, so the experience is completely integrated into the physical space and alterations later in the process are minimal.

An important lesson that we have learned from our experience in bridging the physical and digital comes from the complexities in designing systems that work for all users all the time. We work in environments where the user base and audience include people from all age groups and levels of experience. We design our interactions for these projects to work as smoothly and intuitively as possible for the entirety of a given audience, since the tracking sensors cannot be customized to a specific person’s size or skill level.

We have had success in pushing the boundaries of experience design and usability. For the National Geographic Birds of Paradise "Dance Dance Evolution" experience, the users moves are algorithmically applied to an animated virtual 3D bird model. I took basic skeletal tracking information from a human user and was able to map it to the complex 3D skeleton of a Parotia bird that our in-house 3D modeler built from scratch. The experience made a plantigrade (feet on ground) to digitigrade (walking on toes) translation, and accounted for gestures that operated wings and a feathered skirt. We had to translate each users movement to correlate with the complex movements of the bird wings, so they would be graceful and accurate. Another example, this one for GE’s America Works project, explains how we used gesture-based interaction with Kinects in both horizontal and vertical configurations and is described below.

What value for users did the way you bridged the physical with the digital add to the experience that couldn’t have otherwise been achieved?: 

For the Kennedy Space Center Space Shuttle Atlantis Attraction, visitors engage with a physical system that is configured very closely to the arrangement of the actual shuttle cockpit or enter a dark isolated space that simulates being inside a helmet on a spacewalk. In these experiences, visitors get a simulated, hands-on feel for what Astronauts deal with as they, for example, navigate with only limited viewing angles out of the cockpit windows or rely on CCTV cameras. I developed the 3D simulation software visible in the “windows” to replicate virtual views of the shuttle, space station, Earth and stars the astronauts might be looking at in perspective.

The National Geographic "Dance Dance Evolution" experience was an interactive exhibit that allowed visitors to control virtual 3D Parotia Birds, and replicate moves from the intricate mating dances the birds engage in while trying to attract a mate. Two visitors "dance" against one another simultaneously as the virtual birds reflect their moves through gesture tracking. The users are tracked for how well they perform and surrounded by fellow visitors that are able to rate how well they dance by voting for one or the other. Spatially, the physical set up is similar to how two male Parotia birds dance in a jungle clearing while female birds observe from branches above. The GE America Works project was meant to immerse event-goers in an interactive environment that prompted discussions and knowledge sharing with GE employees to share stories, statistics, and product information with colleagues, politicians, journalists, customers, and the general public.