Real-time Auralization System for Virtual Spaces and On-line Meetings

Stanford researchers have patented a real-time auralization-reverberation system (CAVIAR - Chamber for Augmented Virtual and Interactive Audio Realities) for providing immersive and interactive audio environments. From living rooms to concert halls, this technology can be used in a variety of settings, and for a range of situations including music making, theatrical performances, gaming, conferences etc., at a single location or even across multiple networked locations. The system creates auralizations using standard audio speakers and microphones combined with a novel type of feedback cancellation, transporting audience members and performers to another acoustic space.

The team has produced auralizations that allow people to interact with virtual reconstructions of the Hagia Sophia, prehistoric cave formations in France, the Grand Mosque in Isfahan, Stanford's Memorial Church, and many other spectacular spaces. Outputs from the research team include the first ensemble album recorded in live virtual acoustics, "Lost Voices of Hagia Sophia", which reached No. 1 on Billboard's Traditional Classical Album Chart for three weeks in February and March, 2020 and many public demonstrations through the "Seeking the Acoustic Signature of Transcendence" and the "Sound, Space, and Sensing the Unfathomable" research projects led by Professor Jonathan Berger.

Since March of 2020, in the wake of Covid-19, CAVIAR has been adapted for use with networked systems users to share an acoustic space (see Stanford Docket S18-504B). Applications for this variant of the technology include online meetings, conference calls, and classrooms, networked gaming, and multiple musicians/actors performing as a single ensemble at multiple remote local sites.

Stanford Docket S24-358 further improves the performance by CAVIAR's calibration methods, further limiting the possibility of feedback when simulating extremely reverberant spaces, mitigating against temperature or humidity fluctuations, and by enhancing the simulated acoustics for quiet sound sources.

Stage of Development - Multiple Prototypes:

• Demonstrated system and method at Department of Music and the Center for Computer Research in Music and Acoustics (CCRMA) Open House, Stanford University in March 2019.
• Used system and method to study musician interactions with acoustic spaces at CCRMA, May 2019.
• Icons of Sound: Cappella Romana in a virtual Hagia Sophia - Cherubic Hymn in Mode 1
• Documentary Film: The Voice of Hagia Sophia