Speechin smart necklace recognizes the silent speech of its owner

Last year, Professor Cheng Zhang from Cornell University presented a prototype of a “smart necklace” NeckFace, which skillfully analyzes the facial expressions of the owner. It had a number of drawbacks, so the scientist developed a new version called “Speechin”. This device is able to recognize the owner’s speech, but for this it does not need to see the speaker’s face.

Zhang’s main task was to preserve the privacy of the Speechin user. It is assumed that the gadget will be used to covertly transmit voice commands to a smartphone where a loud voice sounds inappropriate – in a library, during business meetings, in a crowd, etc. Also, the device does not see the user’s face, even his lips, it reads data on the movement of the chin.

Using the principles of machine learning, Professor Zhang, with the help of 20 volunteers, taught Speechin to recognize basic words and phrases. Half of the team spoke English, the second in Chinese, while the accuracy of speech recognition in the first group reached 90.5%, and the second 91.6%. But immediately a significant drawback was revealed – it turned out to be almost impossible to recognize speech on the move, different gait forced the heads of the speakers to move in an unpredictable way.

As additional features of using Speechin, cases are indicated when it is too noisy around and the smartphone cannot make out the owner’s speech. Also, such devices will help those who have lost their voice during illness or due to injury.