[ image ] Create New Art Expression through Technologies Which Accommodate People's Needs

Features / Creating Value for the Next Generation 03
Create New Art Expression through Technologies Which Accommodate People’s Needs

In our long history, Yamaha has endeavored on a day-to-day basis to accumulate research on how we can best deliver the joys of sound and music to people all over the world, and we have been developing our own technologies to this end. By proposing Yamaha’s new value, the opportunity for anyone and everyone to enjoy music, we provide many people the happiness of playing music. As a part of this, in terms of universal design, Yamaha proposes to develop content and infrastructures that enable more people to experience music. As a participant of the Center of Innovation (COI), Tokyo University of the Arts since 2015, we have cooperated with their COI Program which aims at foreseeing and resolving future social issues with our technologies.

[ image ] Point of value creation, Universal design/Succession and
development of
music culture

Overview

Pursuing New Potential of Art by Business-Academia Collaboration

COI STREAM^* was launched by Ministry of Education, Culture, Sports, Science and Technology, Japan (MEXT) and Japan Science and Technology Agency (JST) in fiscal 2014. Holding a vision for the ideal future society and lifestyle, this project aims to identify innovative research and development issues with a 10-year outlook. Upon doing this, the COI STREAM removes barriers of existing fields and organizational walls in order to promote practical applications from the fundamental research level. The COI STREAM intends to provide a business-academia collaboration platform to create innovation that could not be achieved independently as either a company or a university.
One of the sites that promotes this program is the Center of Innovation, Tokyo University of the Arts where Yamaha is a participant. This center attempts to establish a society filled with not only material richness but also spiritual happiness, and to develop abundant cultural contents for the next generation’s infrastructure by introducing “research to cultivate culture” themed on “Synesthetic innovation using art and science technologies that create emotion,” and “research to foster the human spirit” to realize a society where all people live in harmony.”
In this center, an event called the Geidai Arts Special−Disability & Arts is held annually to provide a space to enjoy art and music regardless of disability as well as seek the potential of art in the context of modern society. Yamaha has worked with this event since 2015 and supported people who have disabilities resulting in hand, foot or hearing difficulties to express their music performance better by leveraging our own technologies. Moreover, we’re working to create an innovation using AI by maximizing this event as an opportunity.

COI STREAM: Center of Innovation Science and Technology based Radical Innovation and Entrepreneurship Program

Point1 Universal Design

Supporting Music Expression of People with Disability

[ image ] Image of the performance support system — Image of the performance support system

Automatic Support of Hand and Foot Operation along with the Performance

Geidai Arts Special 2015—Disability & Arts was held in December 2015. At the concert held during this event, a performance support system developed with Yamaha as the core developer assisted a piano performance by students of a special support school with hand or foot disability, and received a high level of attention.
This system combines Disklavier™^*1, an auto playback function piano, with following performance technologies^*2. It supports performance by automatically adding the music of the left-hand portion with the music of Disklavier played by the right hand. The system enables players with hand or foot disabilities to play piano by themselves more freely. Supporting people by machine expands the possibility of music expression and helps to make the dream of a high school student to play their favorite song come true.

1 Yamaha acoustic piano with automatic performance function that correctly records the performance information and replay.
2 Technologies to analyze the performance in real time and identify the playing part on the score. It is possible to automatically play accompaniment along with the performance, flipping score, and synchronize images.

[ image ] Nocturne of Chopin presented using the performance support system — Nocturne of Chopin presented using the performance support system

[ image ] A concert attracting much attention — A concert attracting much attention

Visualizing Drum Performance Loudness

At the concert in Geidai Arts Special 2016—Disability & Arts, elementary school students who have hearing difficulties were challenged to play drum. Yamaha Corporation, Tokyo University of the Arts COI and Special Needs Education School for the Deaf, University of Tsukuba developed a performance support system for students who have difficulty perceiving the volume of drum.
This system detects drum volume and shows loudness along with the score on iPad in real time, so students can play the drum while monitoring the volume on the iPad screen. When they played with Geidai Philharmonia Orchestra, they produced wonderful music by beating pianissimo to fortissimo correctly.

[ image ] Performance support system that visualizes loudness — Performance support system that visualizes loudness

[ image ] Photo (left): Students of Special Needs Education School for the Deaf, University of Tsukuba played with the Orchestra. (right): Concert snare drum and performance support system. Photo: Taira Tairadate, Tokyo University of the Arts — Photo (left): Students of Special Needs Education School for the Deaf, University of Tsukuba played with the Orchestra. (right): Concert snare drum and performance support system. Photo: Taira Tairadate, Tokyo University of the Arts

Comments

The Children’s Ability Was Drawn Out

By being able to rehearse many times for the performance, the orchestra and the students could play ensemble in perfect order at the concert. The children’s learning ability was incredible, and I think it was made possible by the performance support system as this enabled the children to recognize their own sound visually.

[ photo ] Oko Arai — Oko AraiProject Associate Professor of the “Research on disabilities and expressions” group
The Center of Innovation, Tokyo University of the Arts

They Shared a “Common Language”

When I listened to perfectly synchronized sounds and precise pianissimo, I was touched beyond words. Thanks to the performance support system, the students were able to play expressive music and all of the performers shared a “common language.” Our goal was achieved.

[ photo ] Yukiyo Takahashi — Yukiyo TakahashiProject Researcher of the “Research on Disabilities and Expressions” group
The Center of Innovation, Tokyo University of the Arts

It Led to Autonomous Learning

Thanks to the developed system, the students were able to recognize the strength of their own beating sound, and it led to independent learning. Likewise, it allowed them to see the sound volume objectively by comparing the volume produced by their friends around them.

[ photo ] Kayoko Yamamoto — Kayoko Yamamoto
Music teacher of Special Needs Education School for the Deaf
University of Tsukuba

Point2 Succession and Development of Music Culture

Challenging to New Music Expression by Using AI

Inspired by the Geidai Arts Special, Yamaha Corporation supports the COI Programme at the Tokyo University of the Arts by tackling with various needs through technology.
One example is an achievement of music expression that transcends space and time. To accomplish this, we developed “AI Ensemble Technology” that can understand the music played by human and play automatically along with the performance. By combining this system with Disklavier™, we enabled a music ensemble performance featuring the Scharoun Ensemble of the Berliner Philharmoniker and a recuperated performance of the late Sviatoslav Richter, a legendary pianist of the 20th century.
Moving forward, Yamaha will explore new possibilities with our technologies and products through further research, development, and technical cooperation with other organizations, and we will contribute to the creation and preservation musical culture for the future.

[ image ] AI Ensemble Technology developed by Yamaha — AI Ensemble Technology developed by Yamaha

[ image ] Joint concert with Scharoun Ensemble of the Berliner Philharmoniker — Joint concert with Scharoun Ensemble of the Berliner Philharmoniker

Features

[ Image ] Enriching Education in Vietnamese Schools through the Introduction of Instrumental Music Education

Features / Creating Value for the Next Generation 03Create New Art Expression through Technologies Which Accommodate People’s Needs