Decoupling Physical and Virtual Spaces in Co-Located Collaborative Mixed Reality Instruments with gRAinyCloud

Pierrick Uro; Florent Berthaut; Thomas Pietrzak; Marcelo Wanderley

Decoupling Physical and Virtual Spaces in Co-Located Collaborative Mixed Reality Instruments with gRAinyCloud
Image credit: Pierrick Uro; Florent Berthaut; Thomas Pietrzak; Marcelo Wanderley

Abstract:

Collaborative co-located Mixed Reality musical instruments combine some of the expressive opportunities of 3D interaction and communication and cooperation of physical multi-user instruments. However in existing instruments, the fixed coupling between the virtual and physical environments constrains the affordances brought by Mixed Reality, such as per-musician free navigation in or multi-scale control of virtual structures. We designed gRAinyCloud, as a way to reintegrate these lost affordances to a co-located instrument. It allows for the expressive exploration of a set of sounds represented by a virtual structure of shapes placed in the physical space and shared between musicians. Above all, gRAinyCloud enables each musician to freely manipulate their own viewpoint, changing its scale, position and rotation, effectively decoupling the physical and virtual spaces, and to switch between self, other’s and absolute viewpoint while playing. We describe the implementation of this decoupling of spaces and analyse its uses and implications for collective musical expression, by relying on a first-person approach.