By Daniel Vlasic, Rolf Adelsberger, Giovanni Vannucci, John Barnwell, Markus Gross, Wojciech Matusik, Jovan Popovi´c
Commercial motion-capture systems produce excellent in-studio reconstructions, but offer no comparable solution for acquisition in everyday environments. We present a system for acquiring motions almost anywhere. This wearable system gathers ultrasonic time-of-flight and inertial measurements with a set of inexpensive miniature sensors worn on the garment. After recording, the information is combined using an Extended Kalman Filter to reconstruct joint configurations of a body. Experimental results show that even motions that are traditionally difficult to acquire are recorded with ease within their natural settings. Although our prototype does not reliably recover the global transformation, we show that the resulting motions are visually similar to the original ones, and that the combined acoustic and inertial system reduces the drift commonly observed in purely inertial systems. Our final results suggest that this system could become a versatile input device for a variety of augmented-reality applications.
PuppetShow is a system that allows people use there bodies to control virtual characters. Puppet show uses a webcam to track the movements of a puppet and translates the gestures into bone movements in a game engine. Puppet Show’s goal is remove confusing middleware (keyboards and mice) from the player interface and allow the users to use their body’s natural expressiveness to convey themselves in virtual environments.
PuppetShow was built with Processing and runs within the Unreal Runtime using custom characters. Currently PuppetShow is in the late alpha phases and can track any flat color puppet and relay the actions two different specialized characters: a duck and a panda. Currently the PuppetShow team is integrating the complete PuppetShow animation system into the standard Unreal skeletal system. In the near future PuppetShow will be able to control any Unreal character through the PuppetShow interface.
How it Works
PuppetShow in truth is actually two separate systems that communicate over a TCP link: the puppet tracker and the Unreal puppet animation system. Below is the overall flow diagram of how PuppetShow works.
Puppet tracking is accomplished by using a puppet, a webcam, and the PuppetShow color tracking software. The color tracking software uses a refined blob tracking algorithm to track solid colors on a puppet. Since the tracking is based solely on color almost anything can act as a puppet: origami, gloves, socks, shirts and so forth! The tracking color can be set easily with the dynamic color calibration system. Based on a predetermined scheme, the tracking software then takes the blobs and translates them into usable information which is subsequently sent to the Unreal Runtime running the PuppetShow animation system.
The PuppetShow Mutator takes in values from the color tracker and translates them into bone translations and rotations. These are then applied to the character controlled 3D mesh (in the above diagram: a ducky!) which is subsequently displayed on the screen. An important note: PuppetShow does not interfere with the standard input devices of Unreal. This allows players to move and manipulate their characters as they would usually be able to.
Sensisphere is a hemispherical screen with an innovative user interface. It is integrated into the surrounding architecture. On this “porthole” with a diameter of about one meter many different interactive, multimedia-based contents can be experienced. by reamatube
BVH – skeletal data
;){kind=link}