Innermost Echoes: Integrating Real-Time Physiology into Live Music Performances

Danny Hynds, George Chernyshov, Dingding Zheng, Aoi Uyama, Juling Li, Kozue Matsumoto, Michael Pogorzhelskiy, Kai Kunze, Jamie A. Ward, Kouta Minamizawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In this paper, we propose a method for utilizing musical artifacts and physiological data as a means for creating a new form of live music experience that is rooted in the physiology of the performers and audience members. By utilizing physiological data (namely Electrodermal Activity (EDA) and Heart Rate Variability (HRV)) and applying this data to musical artifacts including a robotic koto (a traditional 13-string Japanese instrument fitted with solenoids and linear actuators), a Eurorack synthesizer, and Max/MSP software, we aim to develop a new form of semi-improvisational and significantly indeterminate performance practice. It has since evolved into a multi-modal methodology which honors improvisational performance practices and utilizes physiological data which offers both performers and audiences an ever-changing and intimate experience. In our first exploratory phase, we focused on the development of a means for controlling a bespoke robotic koto in conjunction with a Eurorack synthesizer system and Max/MSP software for controlling the incoming data. We integrated a reliance on physiological data to infuse a more directly human elements into this artifact system. This allows a significant portion of the decision-making to be directly controlled by the incoming physiological data in real-time, thereby affording a sense of performativity within this non-living system. Our aim is to continue the development of this method to strike a novel balance between intentionality and impromptu performative results.

Original languageEnglish
Title of host publicationTEI 2024 - Proceedings of the 18th International Conference on Tangible, Embedded, and Embodied Interaction
PublisherAssociation for Computing Machinery
ISBN (Electronic)9798400704024
Publication statusPublished - 2024 Feb 11
Event18th International Conference on Tangible, Embedded, and Embodied Interaction, TEI 2024 - Cork, Ireland
Duration: 2024 Feb 112024 Feb 14

Publication series

NameACM International Conference Proceeding Series


Conference18th International Conference on Tangible, Embedded, and Embodied Interaction, TEI 2024


  • improvisation
  • liveness
  • music composition
  • physiological sensing
  • public art
  • sonic art

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software


Dive into the research topics of 'Innermost Echoes: Integrating Real-Time Physiology into Live Music Performances'. Together they form a unique fingerprint.

Cite this