IMSC's Multi-Channel Experiment

Yima has been used within the Integrated Media Systems Center (IMSC) grand challenge experiment: Remote Multichannel Media Immersion (RMI) experiment.  The goal of RMI is to immerse a participant in such a high fidelity visual and aural virtual presentation that he or she would truly experience the performance. The terms “remote?and “multichannel? arise from the need to transmit many channels of audio and video information over a network.  Our first realization of this concept is shown in the following figure. We chose to recreate the visual and aural sensations that surround a spectator attending a USC football game (we chose the USC homecoming game). At the left, an array of microphones is placed in the football stadium and the audio signal field they measure is recorded.  Simultaneously, panoramic video is recorded by a camera system (shown at the bottom center of the scene). The processed audio is reproduced (rendered) in an immersive environment (at the right of the figure) that accurately preserves the audio frequencies and their correct spatial relationships to the listener. Combining synchronized panoramic video with the immersive sound completes the immersive experience.  This project imposes the challenging task of synchronized streaming of 10 channels of audio and 5 channels of video from the Yima server.  In addition, the client of Yima needed to be modified extensively to support simultaneous playback of several streams.

RMI experimental setup