PhD Thesis Enrique Tomás “The Interface Score”
supervised by Prof. Dr. Martin Kaltenbrunner and Prof.Dr. Gerhard Eckel
This artistic thesis investigates the artistic and philosophical consequences of understanding musical instruments as artifacts embodying a score. In other words, it explores the extent to which musical instruments can be perceived as musical scores. This artistic research examines the cultural and technological nature of digital musical instruments and it contributes to a material account of musical notation. It introduces a novel paradigm for tangible interface design called the “tangible score”. It also describes a practice-based research journey performing tangible musical interfaces.
A “Tangible Score” is a tactile interface for musical expression that incorporates a score in its physical shape, surface structure or spatial configuration. Using sound as a continuous input signal, both synthesis and control are available simultaneously through direct manipulation on the engraved patterns of the physical score. Every interface is conceived from a different graphical score that still represents a musical idea but it has been also specially designed for providing a diverse palette of acoustic signals when touched. But more important, the tactile scores define and propose specific gestural behaviors due to the different affordances and constraints of the object in front. Sound is generated through a polyphonic concatenative synthesis driven by a real-time analysis and classification of input signal spectra. Each of the scores is loaded with a specific sound corpus that defines its sonic identity. Thus, “Tangible Score” provides a implicit visual and haptic feedback in addition to its sonic core functionality, making it intuitive and learnable but as well suitable as an interface for musical improvisation and sonic exploration.
But, in which ways is it possible to embody musical works into musical instruments? ‘A Moment of Transition’ is a live electronics work performed with a “tangible score” whose form has been inspired by certain sound gestures. Through the ‘action of the hand’ Tomás’ explores the tactile patterns engraved on the instrument. A machine learning system clusters and recognizes tactile and sonic contents enabling real-time manipulation of digital information previously stored and analyzed in the computer. A generative lighting system augments the paper’s physical materiality while it also inscribes a visual narrative inspiring musical forms. The result of this physical encounter is a live performance where sound, light and interface become both musical notation and instrument.
This performance shows us how much our body is the central actor empowering those ‘moments of transition’ which transform artistic intention into sonic outcome. Our bodies, embedded in a perpetual action and perception loop, explore physical materiality and inform us about the world. But our bodies are noisy. They do not only enable performance but they also sculpt our ´being-in-the-world´. In ‘A Moment of Transition’ interface´s materiality becomes physical notation provoking performace. Tomás´ noisy hands, freely exploring the affordances of this system, finally produce the sound work we perceive.
Technically, touch sensors and contact microphones are used to
capture tactile activity on the paper which has been embossed with a
CNC. This information is chuncked and analyzed in real time extracting
mathematical descriptors (Mel Frequency Cepstral Coefficients). This
information is used to drive a machine learning system resynthesizing
captured gesturality into new sound. This is achieved after ordering and
concatenating chunks of a digital sound archive previously analyzed and
clustered by the same algorithm.
(Produced with the support of ENCAC Network)
This is a thesis in art. Opposed to a thesis on art, such as art history, my research did not have any prescribed methodology. In fact, my artistic production during this PhD may be considered both object and method. As an artist working with technology it was mandatory contextualizing my personal artistic methods. In consequence, this thesis gives answers to the question of how artists can engage artistic research with the field of human-computer interaction (HCI). I defend that the way artists can help HCI is adopting a critical attitude with our research medium -namely tangible and musical interaction design in this PhD- avoiding the instrumentalization of our artistic processes and adopting the format of artistic research.
The notion of the “inherent score” drives the beginning of my research on musical notation. I define it as the material and virtual elements of the instrument shaping and inspiring a player’s performance. This notion gives us language to describe how the instrument mediates embodiment at the exact moment of performance. This is coherent with many improvisers’ intuition affirming that the instrument is the score of what they play. I propose that it is possible to design new interfaces for musical expression emphasizing their “inherent score”.
For exemplifying this option, I have built and performed a series of digital musical instruments called “Tangible Scores”. They are the center of my artistic practice during this doctorate.This thesis contributes to a vision of digital musical instruments as symbolic machines. In my opinion,the principal characteristic of digital music instruments is their symbolic apparatus. The apparatus always features a program defining all possible musical realizations. For dealing with the complexities of the apparatus, designers have to deal with representations. For this reason, the representational dimension of musical interfaces has been addressed in this thesis. I present a vision about musical interfaces as a collection of related texts and symbols with a given arbitrary meaning. As I show in this thesis, they radically mediate our embodiment with the instrument. For this reason, I contribute towards a non-linguistic and non-representational approach to performing with musical interfaces.
Materiality is another important field of study in this thesis. I contribute to a materialistic accountof musical interfaces defending that matter must not be only understood as the mere support for an interface’s functionality. My thesis asserts that matter is an active agent able to inspire, provoke and shape performances. Therefore, my research aims at developing a new materialist sensibility at tangible and musical interaction design.Finally, this thesis addresses the cultural and political dimensions of musical interface design. I defend that the cultural always re-configures and redefines the technical in musical interface design. Politically, musical interfaces are never transparent technologies. A full range of cultural and political values are scripted into our musical interfaces. They are good examples of cultural interfaces.
In conclusion, this is a thesis dedicated to the study of the issues of musical interface design from an artistic standpoint. The field of study is highly interdisciplinary and cultural studies, technology and design-among many other disciplines- merged at my practice. For this reason, the background of this thesis is art,music, engineering and philosophy.
Lecture @IRCAM: Towards Non-Linguistic Writing for Music – A Performative Approach (New Notations Symposium)
Some photos of Tangible Scores:
Doctoral Thesis 2018: The Interface-Score
NIME’s 2014 paper: Tangible Scores: shaping the Inherent Instrument Score
TEI’s 2016 paper: Tangible Scores: an Inherent Score Installation
TENOR’s 2016 paper: Instruments as Score: A Hybrid Approach
Interface Politics Conference 2016 paper: Politics of Musical Interfaces: Ideologies and Digital Disenchantment
2017 – Informatics Journal (MDPI): How the Arts Can Help Tangible Interaction Design: A Critical Re-orientation
ICMC + SMC 2014 Concert Program Notes: Tangible Scores Concert ICMC – SMC:Tangible Scores Concert ICMC – SMCD
http://cdm.link/2017/02/touch-feel-score/Article @ CDM and Adafruit Blog
ACM – Interactions: Review
Sul Ponticello Magazine: The Overloaded Score (article in Spanish):
La Sexta TV: Sónar 2014 Report
WAZ German Newspaper: Tangible Scores at Blaues Rauschen (review)
Some concerts and presentations of Tangible Scores:
– ICMC+SMC Conference Athens 2014
– Sonar Festival Barcelona 2014