Entangling with Light and Shadow: layers of interaction with the pattern organ

Jasmine Butt; Benedict Gaster; Nathan Renney; Maisie Palmer

Entangling with Light and Shadow: layers of interaction with the pattern organ
Image credit: Jasmine Butt; Benedict Gaster; Nathan Renney; Maisie Palmer

Abstract:

This paper explores the design and use of a camera-based digital musical instrument as a thinking tool for considering entangled, post-human perspectives. The design of the pattern organ, inspired by experimental optical sound-on-film practices, employs a method of visual-to-audio synthesis that responds closely to the material behaviours captured by its camera input. Drawing on findings from exploratory workshops and short material experiments, we describe how interactions emerge and are shaped by both the physical configuration of the instrument and the material behaviours captured by its camera. We consider how frugal mappings and the ‘rawness’ of data can give rise to instruments whose inputs remain open to material complexity, extending the sound engine beyond their enclosures. In the case of the pattern organ, this complexity emerges through overlapping and interfering interactions, where structural forms, human influence, light, shadows, lens distortions, and system quirks all contribute to the shifting harmonic content of the wavetable. We reflect on the instrument as a fluid assemblage, composed of human and non-human entanglements, encouraging us to think beyond traditional notions of human-centred control.