David Loberg Code is Professor of Music at the Irving S. Gilmore School of Music, Western Michigan University.
Duration
3 months, 5 April to 30 June 2027
Host at the University of Passau
Prof. Dr. Tomas Sauer
(Chair of Digital Image Processing)
Digital sonification is the mapping of data from one domain (e.g., visual images, biosignals, climate data) onto parameters of sound such as pitch, duration, loudness, timbre, or location. It is the auditory equivalent to data visualization and likewise can serve both analytic and artistic goals. The analytic function of cross-modal data representation is to make it easier to detect specific features of patterns in the source dataset. The goal of artistic data representation is to create an aesthetically pleasing or compelling work of art that is derived from the source data. This project will focus on two primary areas of research. The first involves the sonification of biosignals from EEG devices in collaboration with the Human-Machine Interactions lab. EEG data is well-suited to sonification, because it is basically the amplitudes of low-frequency waves as they unfold in time. We hope to not only use this technique to highlight meaningful data for diagnostic purposes, but to generate musical signature tunes of an individual's brainwave activity. Our second area of investigation is the mapping of color to sound primarily for the use of blind individuals. The goal is not to replicate the phenomenon of pitch-to-color synesthesia, but rather to provide a tactile interface with which someone could scan a color image and be provided with audio feedback that delineates the colors therein. Imagine being able to audibly scan an image of the Dreiflüsseeck and be able to hear the colors of the water in different timbres as they merge together.
