Breakthrough in Hearing Aids: Now They See Sound Source

A team of scientists from the National University of Chang Kung, led by Professor Chun Du, has developed a prototype of a hearing apparatus that aids people with hearing impairments in determining the source of sound more accurately. This device combines various advanced technologies, including computer vision, specialized algorithms, and microphone arrays. The team aims to integrate this module into the daily lives of older patients with hearing impairments to enhance their quality of life.

The device consists of an innovative two-layer microphone structure placed on the ears. Additionally, it includes a device with a video camera and computer vision technology worn on the neck. The camera utilizes algorithms to recognize faces within its field of view, helping identify the direction of sound. When the speaker is out of the camera’s visibility zone, an alternative algorithm utilizes the analysis of sound wave angles and time delays to determine the location of the sound.

To achieve more accurate sound direction determination, a specialized mixing algorithm adjusts the sound entering the headphones and regulates the volume to create an immersive effect.

In testing with 30 patients with hearing impairments, the device exhibited excellent results. Participants successfully identified the source of sound with an accuracy of 94% or higher using computer vision at conversation distances (up to 160 cm). Even without the camera, relying solely on microphone data, accuracy exceeded 90%.

Furthermore, a specialized mixing algorithm effectively balances the volume of the left and right sound channels in the headphones, enabling users to better determine the direction of the sound source. In a separate study involving elderly participants and combined technology, there were 100% successful attempts to determine sound direction in clinical tests.

However, Professor Du highlights that the video camera’s viewing angle of 75 degrees is inadequate compared to human vision (120 degrees). The team plans to incorporate wide-angle lenses or additional cameras in the future to expand the recognition zone and enhance the device’s suitability for everyday use.

The team at Chang Kung University is interested in further technological advancements, particularly integrating facial recognition and voice recognition functions to automatically identify the speaker within the camera’s field of view. This will significantly improve communication comfort for users. The team is currently in discussions with potential partners regarding the commercialization of their development.