Vision     Experiments     Approach     People     Publications     Related Links  

2006, Jun 6 California StreamIN
Leila Vaziri and Rob Parke ChangHyun Kim

Movie [Win Media] (24.8 MB)

In a collaboration between Stanford's SoundWIRE Group and USC's DIP Team, graduate students ChangHyun Kim (CCRMA, Stanford), Rob Parke and Leila Vaziri (MuCoaCo, USC), with the help of Sakire Arslan and Qin Min (DRML, USC), successfully put together and demonstrated the first phase of an extended distributed immersive performance experiment between Stanford and USC. The two-way low-latency audio-only experiment between the two sites had ChangHyun and Leila learn, practice, and perform Mozart's "Larghetto und Allegro" over the Internet.
Gallery and links: http://imsc.usc.edu/dip/CaliforniaStreamIN
ChangHyun's report: http://ccrma.stanford.edu/~flipflop/220c/220c_webpage.html

2004, Sep 28 Internet2 Concert Live HD Streaming
Miro Quartet Virtual Miro
The Miró Quartet (Daniel Ching, Sandy Yamamoto, Joshua Gindele and John Largess) performs in Bates Recital Hall, UT Austin, with half the audience on location, and other half in McCullough Hall (150 yards away). The concert is rendered in HD video and 10.2-channel immersive audio. Evaluations of immersion and presence in real and virtual performances being performed with UTexas.
IMSC news release: imsc.usc.edu/news/releases/i2_040929.html
UTexas news: www.utexas.edu/features/archive/2004/internet.html
2004 May-July DIP v.2: Tosheff Piano Duo, duet with audio delay
Two-way baseline user studies to measure and document qualitatively and quantitatively effects of delay and other variables on immersion, usability, and quality in the DIP scenario. Tosheff Piano Duo (Vely Stoyanova and Ilia Tosheff) on Yamaha P80 weighted action keyboards in same room facing each other, performing movements from Poulenc's Sonata for Piano Four-Hands. Recordings of duo performing with audio latency captured on May 19,21 and Jul 22,23, for four sets of conditions:
    A -- first time players perform under delayed conditions.
    B -- symmetry test (player 1 and 2 swap parts).
    C -- players practice to compensate for delay.
    D -- players perform with both partner and self delayed.
The Tosheff Piano Duo: www.tosheffpianoduo.com
See Publications for descriptions of Research Findings. For example,
2004, January Two-way Live HD Streaming Experiments
(Los Angeles, Hawaii, Miami)

2003, June DIP v.1: Distributed Duet (audio and video)
Distributed Immersive Performance Distributed Immersive Performance

Movie 1 [Win Media] [Quicktime] Movie 2 [Win Media] [Quicktime]

EXPERIMENT SETUP: NSF site visit and annual SAB/BOC meeting. Dennis Thurmond (accordion) at PHE106 and Elaine Chew (piano) at RMH106, performing Piazzolla's Le Grand Tango to audience co-located with accordionist at PHE106. Chew has earphone and video monitor, at audience site are 10.2-channel immersive audio and large screen HD image. NTSC-resolution video, 31 Mb/s DV, software decode (one-way latency 110ms + <5ms (compression + network)). Uncompressed audio, 15 or more channels at 1 Mb/s each (one-way latency <10ms + <5ms (processing + network)). Piazzolla's piece had an approximate tempo of 120 bpm; granularity of events at 15th note level gave an average IOI of 125 ms.
RESEARCH FINDINGS: Video was unusable as source of of cues for synchronization due to delays. Audio (<<50ms RT delay) was used for cueing. Musicians compensated for delay by anticipating each others' actions and scaling back on spontaneity; some artistic licence was exercised to "make ends meet." Co-location of audience with one musician caused imbalance in control.
IMSC newsletter: imsc.usc.edu/newsletter/pdfs/IMSC_News_03_10.pdf (p.3)
See Publications for overview of experiments to date, e.g. [ ETP2003 ]
2003, Jan 19 Remote Masterclass (audio and video)
Ron Leonard in Masterclass
EXPERIMENT SETUP: Ron Leonard, renowned LA Phil cellist, conducts masterclass with students in New World Symphony. Teacher located in PHE106, with NWS students in Miami, Florida. At Leonard's site is 10.2-channel immersive audio by Kyriakakis and Holman. Used off-the-shelf video software/hardware (Star Valley MPEG2 codecs), large delays.
RESEARCH FINDINGS: Teacher acknowledged improved presence with immersive audio: "student was really there."
2003, January Recording from streams
2002, Dec 27 Distributed Duet (immersive audio only)
EXPERIMENT SETUP: Elaine Chew (keyboard) in PHE106 with stereo audio, Wilson Hsieh (viola) in EEB with Kyriakakis and Holman's 10.2-channel immersive audio. Low-latency audio streaming software created by Papadopoulos and Sinh actual delay (varying between 0ms and 300ms) controlled using Protools console. Pieces played are Hindemith's Viola and Piano Sonata No.4 in F major Op.11 and Piazzolla's Le Grand Tango.
RESEARCH FINDINGS: Latency tolerance dependent on tempo of piece and onset synchronizations after pause, and timbre of instrument. Sense of acoustic presence made more "natural" by 10.2-channel audio. Perspective difference requires recording of experience at both sites.
2002, Oct 29 Cultivating communities: Dance in the Digital Age
Performance event hosted by Internet2, University of Southern California, and Manhattan School of Music; held at USC's Bing Theater. Presentations include distributed performances (dance with music), and a large room RMI demonstration.
Internet2 report: arts.internet2.edu/fall2002-perfevent.html
IMSC newsletter: imsc.usc.edu/newsletter/pdfs/IMSC_News_02_12.pdf
2002, June Remote Media Immersion (RMI)
Initial demonstrations begin.

This material is based upon work supported by the National Science Foundation under Grant No. 0321377