Auditory-Tactile Music Perception

Sound and vibrations are often perceived via the auditory and tactile senses simultaneously, e.g., in a car or during a rock concert. Even in a concert hall or a church, sound can excite suprathreshold vibrations in the ground or seats. If concert recordings are played back through headphones, this vibratory information is missing to date. The same holds true in the majority of cases for reproduction with multimedia or high-fidelity systems.

This thesis extends our understanding of the coupled perception of sound and vibration using the example of auditory-tactile music perception. The capabilities and limitations of both modalities are compared first. Unfortunately, particularly for the perception of vibrations at low levels, only limited knowledge exists to date. Therefore, the frequency discrimination and intensity perception of whole-body vibrations is investigated. The most evident difference between both modalities is the dramatically reduced ability to distinguish between vibration frequencies in the tactile domain. Another important difference is the steeper growth of the perceived magnitude for touch compared to hearing. A new perceptually motivated measurement for the perceived vibration magnitude M is defined to represent human vibration intensity perception. Additionally, cross-modal effects are considered, e.g., the influence of whole-body vibrations on loudness perception. An auditory-tactile loudness illusion is proven.

In the second part of this work, it is investigated whether sound-induced whole-body vibrations influence the quality of a concert experience. The fundamental knowledge gained in the first part, is used to develop various perceptually optimized approaches to generate vibrations from music sequences.