Take a Bow: AI Identifies Violinist Movements
A system identifies violinists' bowing techniques with 94 percent accuracy. Credit: UPF
In playing music, gestures are extremely important, in part because they are directly related to the sound and the expressiveness of the musicians. Today, technology exists that captures movement and is capable of detecting gestural details very precisely.
In a study published this month of March in Frontiers in Psychology, David Dalmazzo and Rafael Ramírez, members of the Music and Machine Learning Lab of the Music Technology Group (MTG) at the Department of Information and Communication Technologies (DTIC) of UPF, apply artificial intelligence to the automatic classification of violin bow gestures according to the performer’s movement.
“We recorded movement and audio data corresponding to seven representative bow techniques (Détaché, Martelé, Spiccato, Ricochet, Sautillé, Staccato and Bariolage) performed by a professional violinist. We obtained information about the inertial motion from the right forearm and we synchronized it with the audio recordings”, explain Dalmazzo and Ramírez, authors of the study.
A system identifies violinists’ bowing techniques with 94% accuracy
The data used in this study are available in an online public repository. After extracting the characteristics of the information concerning movement and audio, the researchers trained a system to automatically identify the different bow techniques used in playing the violin. The model can determine the different techniques studied to more than 94% accuracy. The results enable applying this work to a practical learning scenario, in which students of violin can benefit from the feedback provided by the system in real time.
This study was conducted within the framework of the TELMI (Technology Enhanced Learning Performance of Musical Instrument) project. Its purpose is to investigate how technology (sensors, multimodal data, artificial intelligence, and computer systems) can improve the practices of students of music, helping them to focus on the precise development of good practices, especially when incorporating new musical skills.
With the violin as a case study, one of the main goals of the project is to provide students with feedback in real time, as well as allowing them to compare their performances with those of leading experts. “Our findings have already been generalized to other musical instruments and applied in music education environments”, adds Rafael Ramírez, principal investigator of the project.
This article has been republished from materials provided by Pompeu Fabra University. Note: material may have been edited for length and content. For further information, please contact the cited source.
Reference: Dalmazzo, D., & Ramírez, R. (2019). Bowing Gestures Classification in Violin Performance: A Machine Learning Approach. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.00344
Since Darwin's era, scientists have wondered how flightless birds like emus, ostriches, kiwi, cassowaries and others are related, and for decades the assumption was that they must all share a common ancestor who abandoned the skies for a more grounded life. A team of Harvard researchers believes they may now have part of the answer.READ MORE