Physical Representation

The research project “Tangible Signals” investigates the physical representation and haptic feedback control of computer sound data.The creation and manipulation of sounds is fascinating and challenging. The availability and ease of use of microcontrollers, sensors and easily accessible programming environments supported the transition from Graphical User Interfaces (GUI) to Tangible User Interfaces (TUI). But while many NIMEs and TUIs concentrate on enabling the creation and control of digital data, especially the control of musical parameters through the availability of tangible interaction, yet there are only a few interfaces and devices which are able to dynamically represent data in a physical way and at the same time offer the possibility to control or change this data representation and thus the data itself. The physical representation and modification of digital sound data including haptic feedback is the core of this research work Tangible Signals. The audience is at first visually impaired musicians who are limited in their work with GUIs, but also sighted musicians who want to integrate a physical level into their creative process. In addition, this research may also be of interest to other disciplines that benefit from physical representation and modification of data.

Research Questions:

• What are the benefits of tangible musical interaction for visually impaired and blind musicians?
• How does tangible musical interaction contributes to the artistic expression and musical performance of visually impaired and blind musicians?

Accessible Music Software

For creating computer-based music with children and adults with VI, the simplified web-based live code environment WELLE was developed. WELLE serves also as a tool and interface platform for connecting the various tangible signal prototypes to computer generated sounds and sequences.

Live demo at: https://welle.live

Interactive Physical Sound Displays

Three tangible user interface prototypes were developed, that use sensor technology in combination with mechanical actuators to provide an interactive physical display of sound and music data for people with visual impairments. The interfaces can be used either separately or combined as input and output devices utilizing pin-based, string-based and wheel-based interaction elements.

Demonstration of Sonic Interactions at TEI ‘21

– – – – – –

* This PhD project is funded with the DOC fellowship by the Austrian Academy of Sciences, starting June 2019.

* In collaboration with the Federal Institute of the Blind, Vienna.