This project aims to investigate human capacity to decode the communication delivered via body action in virtual human representations (avatars/agents). Previously, IMSC UCS research has examined the factors that contribute to successful detection, by human users, of emotional facial expressions conveyed by avatars actuated using Performance Driven Facial Animation. The current work has expanded to investigate similar issues for full body non-verbal expression of emotion.

The knowledge gained from these projects will drive the integration of UCS methodologies in the design, development and evaluation of IMS that incorporates gestural interaction with both standard magnetic tracking and vision-based systems. This work could also have wider generalizable value for creating the enabling technology required for development of multimodal and perceptual user interfaces (PUIs). The goal here is to supplement current human-computer input modalities, such as mouse, keyboard, or cumbersome tracking devices and data gloves, with multimodal interaction and PUI approaches to produce an immersive, coupled visual and audio environment. The ultimate interface may be one which leverages human natural abilities, as well as our tendency to interact with technology in a social manner, in order to model humancomputer interaction after human-human interaction. Recent discussions on the incremental value of PUIs over traditional GUIs suggest that more sophisticated forms of bi-directional interaction between a computational device and the human user has the potential to produce a more naturalistic engagement between these two complex "systems". This enhanced engagement is anticipated to lead to better HCI for a wider range of users across age, gender, ability levels and across media types.

NSF Report (Year 7)
NSF Report (Year 8)

Poster

The goal of this project is to develop a modular framework for vision-based intelligent environments (Easy Living, Smart Rooms, Intelligent Houses). A generic configuration of the system includes five modules: a panoramic video input component that uses an electronic pan-tilt-zoom camera array, a background learning and foreground extraction component, a tracking component and an interpretation component.

A challenging application of such device is the use of multiple GlobeAll sensors for capturing the real 3D body motion from video streams. We show that an integration of the 2D views captured by a set of two or more GlobeAll camera systems provides an accurate 3D representation of the human body and an efficient approach for disambiguating human motion.

IMSC's 2001 NSF Report
More Info
Laboratory

A real time 3D human body reconstruction is performed from couple of synchronized cameras. The system uses back- projected Infrared lights for an accurate detection of the people within the field of view. Silhouettes of the detected regions are extracted and registered allowing a 3D reconstruction of the human body using Generalized Cylinders. An articulated body model is fitted to the 3D data and tracked over time using a particle filtering method (only pre- recorded results).

Laboratory
More Info