-Voice recognition software reads your brain waves.

Posted: November 13, 2008 in 2008, Articles
Tags: , ,

New Scientist

Mind-reading software developed in the Netherlands can decipher the sounds being spoken to a person, and even who is saying them, from scans of the listener’s brain.

To train the software, neuroscientists used functional magnetic resonance imaging (fMRI) to track the brain activity of 7 people while they listened to three different speakers saying simple vowel sounds.

The team found that each speaker and each sound created a distinctive “neural fingerprint” in a listener’s auditory cortex, the brain region that deals with hearing.

This fingerprint was used to create rules that could decode future activity and determine both who is being listened to, and what they are saying.

Tell-all brain

“We have [created] a sort of speech-recognition device which is completely based on the brain activity of the listener,” explained Elia Formisano of Maastricht University, who led the group.

The team hopes to match recent advances in using fMRI to identify what a person is looking at from their brain activity. Until now, the best mind-reading feats achieved for auditory brain activity extended only to differentiating between different categories of sounds, such as human voices versus animal cries.

“This is the first study in which we can really distinguish two human voices, or two specific sounds,” Formisano told New Scientist.

Hello, I’m listening

In the course of making that possible, the team made new discoveries about how the brain processes speech.

They found that, whatever sound a person makes, they trigger the same voice fingerprint in the brain. Likewise, a given vowel sound produces the same fingerprint, independent of the speaker

This makes it possible to have software recognise combinations of speakers and sounds that it has not encountered before. It should be possible to teach the system how to recognise all the component sounds of speech, and then recognise full words, Formisano says.

“Vowel sounds are not meaningful, but they are language,” he says. “These are the building blocks of language.”

Noisy world

The group are now working to make the system robust enough to recognise more complex sounds without training, and in a noisy environment.

“It’s interesting to see fMRI techniques extended to the auditory cortex,” said Kendrick Kay of the University of California, Berkeley, who studies the brain’s visual systems using similar methods and earlier this year created a system that decodes brain activity to reveal what a person is looking at.

The fact that the Netherlands team has shown that the brain deals with the categorisation of speakers and sounds differently is particularly significant, he says.

Office helper

The new approach could help improve voice recognition software, Formisano says, although probably not by leading to a commercially-available system to transcribe conversations from people’s brains.

Instead, the existing mind-reading software will make it possible to break down the human brain’s impressive ability to process sound and voices, even in the face of distracting background noise. Copying those tricks would provide a new way to make voice recognition software much more effective.

Journal reference: Science (DOI: 10.1126/science.1164318)

Advertisements
Comments
  1. Chris Jones says:

    Your short 17 minute piece, “They want your Sole” is really terriffic on scientific, artistic, and political levels. It is a shame that the website itself is put into the spam portions of people’s mail. I had to send it via googling “They want your sole.”

    I assume you are a neuroscientist. I used to be a scientist and like you like Ron Paul and Alex Jones.

    One more thing. Your picture, believe it or not, looks like a hybrid of Bush and Obama. Have you ever been told that before?

    Keep up the good work.

    Regards,

    Chris

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s