Yamaha Tech Showcase in Shibuya

Content Experiences Focused on Voice Quality and Singing Voice

TransVox™ converts a person’s voice quality into that of a completely different individual. At this event, we offered an immersion experience where guests’ words spoken through a microphone were converted almost in real time into the voice of virtual idol Otoha Tsukitsuka. This technology is envisioned not only as a tool for entertainment applications like VTubers, but also as a tool to assist individuals with speech impediments.

Many guests had never encountered such technology before. They were surprised to hear their own voices transformed instantly on the spot, and some even recorded videos of their co-workers enjoying the experience.

• TransVox

Making Specialized Technology More Accessible

An equalizer (EQ) is a device or software that you can use to create the optimal sound quality for your environment. While equalizers are typically used for sound adjustment, many people find them complex and difficult to operate. Yamaha has developed technology that allows users to find their preferred sound quality through intuitive, non-traditional operation, and then suggests the optimal equalizer settings.

Yamaha is also focusing its efforts on researching sound recording technology. The “ViReal Mic” uses a spherical microphone to capture sound from multiple directions. For example, when recording thunder outdoors, it can capture the variations in the intensity of the sound depending on its direction. Similarly, the “RadCapScanner” device focuses on the spatial spread and the variations in directional intensity of sound. It visualizes the variations in intensity of sound radiation within a space for instruments and other sound sources.

The guests were able to listen to actual recordings made with the ViReal Mic. They were amazed by the realism and made comments like, “I want to measure something with this,” and “I want this array.”

• Spatial Audio Technology（ViReal®）

Yamaha’s Continuously Evolving Instrument Design and Development

Yamaha’s proprietary instrument design and development technologies are constantly evolving. For example, research is advancing toward practical applications, including efforts to quantify our impressions of the sounds we hear, and technology that simulates the sound of a guitar after you have specified its design and materials. Furthermore, research into instrument materials is essential for sustainability. Using wood for acoustic guitars as an example, research is advancing on the differences in sound produced by various wood species and efforts to reproduce the sound of rare woods using alternative materials.

The guests provided high praise, such as “The approach to scientifically capture individual sensibility—a value judgment axis that is difficult to visualize or verbalize—was impressive. I also felt a sense of pride in the engineers’ attitude, demonstrating they have thoroughly mastered experiential learning by observing their mentors,” and “It was clear they are actively trying various approaches towards sustainable manufacturing.”

• Vibration and Acoustic Analysis Technology for Acoustic Guitars

Utilizing Technology Such as AI to Make Playing Instruments Even More Enjoyable

Yamaha is also focusing on technologies that enhance the enjoyment of playing musical instruments. For example, the “Saxophone-playing avatar support package” provides a system where an avatar mirrors a saxophone player’s movements. When the player performs on Yamaha’s digital saxophone, sensors built into the instrument capture the movement, and the software synchronizes these motions with an on-screen avatar. It opens up various potential applications, including saxophone performances by VTubers.

Additionally, the “Guitar-playing support system” allows you to enjoy singing while playing the guitar even without knowing chords. Based on the player’s singing voice, AI detects which part is being sung and a special device presses the strings to make the corresponding chords. The singer can play a song even if they do not know the chords, with just one hand strumming the strings. This system was realized through collaboration: Kumamoto University developed the device that holds the chords, while Yamaha developed the AI technology that detects and tracks the part being played.

• AI Music Ensemble Technology

Unique Initiatives Born from Collaboration

In addition to technology, numerous unique initiatives born from collaboration have also emerged.

“Yokohama AI Idol Kurumi Minato Production Experience” allows you the freedom to produce content based on the AI idol Kurumi Minato and the music she sings—adjusting the tone of her singing voice, changing the musical style, or altering the performance environment. We created this new musical experience by combining “Harmony Sensor,” a music-based inquiry learning system from DATTARUJIN Inc., with Yamaha’s vocal synthesis technology. The guests offered concrete suggestions for future applications, such as “This could be combined with our own content to create something interesting” and “This initiative could be the basis for a new business model.”

“sound biotope,” a co-creation project that enhances creativity through sound, is a collaboration with the creative collective Konel Inc. Within “magmabooks,” a new concept from Maruzen Junkudo Bookstores Co., Ltd., we have created a space in which sound eases your thoughts and soothes your mind. Utilizing the principle of suikinkutsu, a technique from Japanese gardens, the space is filled with the sound of dripping water.

Additionally, “Leisure Encyclopedia,” our joint research project with Shizuoka University, represents a new initiative extending beyond the fields of sound and music. While career education for students has become widespread in recent years, much of it tends to focus heavily on work. This project, therefore, puts the spotlight on “leisure” and creates web content and other materials to convey the breadth of life.

We also introduced “MINA Lab,” a Yokohama-based development hub where we collaborate with external partners to advance research in AI-centered music information retrieval technologies. Utilizing the previously mentioned “performance-following AI,” open proof-of-concept experiments are underway, including “Daredemo Piano,” which automatically generates accompaniment for melodies played with one hand. The guests expressed interest in our AI initiatives and the measurement systems under development, with several stating they would like to explore concrete opportunities for collaboration.

• Yokohama AI Idol Kurumi Minato Production Experience
• sound biotope
• Leisure Encyclopedia
• MINA Lab