The AudioVirtualizer is an interactive and generative Virtual Reality environment created as a VR art experience for the HTC Vive Pro or Oculus Rift headsets. A participant explores three different Op-art inspired scenes which reveal different mappings between musical input and visual output. In each scene the user can select from a range of different soundtracks, or live microphone input, to investigate how the visuals respond to the sound. The graphics are based on different drawings and 3D models by Adinda van ’t Klooster, and teleportation between the scenes as well as a within scene navigation allow the user to explore each environment to the full. The audio engine tracks low- and mid-level features in the sound, allowing more complicated visual returns than previous FFT filterbank based audiovisualisers.
The AudioVirtualizer system first launched at PROTO in Gateshead, in September 2019. It was created for SteamVR and runs as a standalone application generated through Unity.
The graphics are generated in response to changes in the sound features and also exist in the landscape via skyboxes that contain high-resolution scanned drawings. Certain elements from the drawings are then repeated in sculptural objects that are generated in response to the sound and move through the landscape in different ways. The drawings have a sense of minimalism and are manipulated digitally to continue their endlessly repeating patterns in the 3D space as seamlessly as possible. The user is able to see how the graphics respond to different sound sources through a built-in menu option that allows them to choose one of four soundtracks or live microphone input. Pressing the menu button on the controller overlays the scene with five sculptural objects, each of these leading to a different sound source when pressed. Selecting the menu button again will remove the options and continue to play the current selection. All graphics and system design are by Adinda van ’t Klooster, machine-listening tech and some Unity programming was done by Nick Collins, with additional assistance with Unity VR programming provided by Nathan Flounders.
There are essentially three different scenes and an entrance space where navigation can be practiced. This app is available on the Viveport store for free here.