An Interactive Learning-aid System for Analytical Comprehension of Music by Highlighting Orchestral Score in Colors

This paper describes an interactive learning-aid system for analytical comprehension of music by highlighting orchestral score in colors, and classifies and evaluates the learning process on the system. An orchestral music is composed to integrate many instrumental parts, and musicians have to be proficient in reading the score analytically in order to understand its multifaceted structure. However, many people often face difficulty in comprehending its musical structure: Some intermediate performers can read and perform their own part, but cannot understand the role of each part in the assembled whole. In order to solve this problem, our conventional paper proposes an interactive supportive system called ScoreIlluminator that enables musicians (and non-musicians) to easily represent how he or she recognizes an orchestral music, e.g. the differentiation of melody parts from the others, and the similarity across instrumental parts. ScoreIlluminator clusters the parts from an orchestral score according to their roles in the whole, and displays the clusters on the score by assigning a color to each cluster. The users can manipulate the clustering parameters with the user interface of the system. The system employs two major design concepts. One is “colored notation” and the other is “directability”. The “colored notation” visualizes the roles and the relations between parts, which are estimated by the system. The estimation is based on the similarity metric of four musical features: rhythmic activity, sonic richness, melodic activity and consonance activity. Using these metrics, clustering phase is conducted using an unsupervised learning algorithm (k-means algorithm). Our system provides the “directability” with an interactive interface in which subjects can freely manipulate parameter settings and see the change in score-highliting in real-time. In this process, users learn the role of parts and the relationship between parts and explore multifaceted interpretations of the music. To verify the effectiveness of the system, we conducted a user-experience experiment with four intermediate musicians. The musicians showed various kinds of progress in interpreting the score. With the episodes from the experiment, we discuss how the system encouraged subject's analytic skill in orchestral-score reading and music listening. keywords: learning-aid system, score reading, musical comprehension, musical analysis.