Silent speech interface
Encyclopedia
Silent speech interface is a device that allows speech communication without using the sound made when people vocalize their speech sounds. As such it is a type of electronic lip reading
. It works by the computer identifying the phoneme
s that an individual pronounces from nonauditory sources of information about their speech movement
s. These are then used to recreate the speech
using speech synthesis
.
and optical camera input of tongue
and lip
movements. The detection of speech movements by electromyography
of speech articulator
muscles and the larynx
is another technique. Another source of information are the vocal tract
resonance signals that get transmitted through bone conduction
called non-audible murmurs.
They have also been created as a brain–computer interface using brain activity in the motor cortex
obtained from intracortical microelectrode
s.
needed for audible speech such as after laryngectomies
. Another use is for communication when speech is masked by background noise
or distorted by self-contained breathing apparatus. A further practical use is where a need exists for silent communication such as when privacy is required in a public place, or hands-free data silent transmission is needed during a military
or security operation.
In 2002, the Japanese company NTT DoCoMo
announced it had created a silent mobile phone
using electromyography and imaging of lip movement. ‘‘The spur to developing such a phone,” the company said, ‘‘was ridding public places of noise,” adding that, ‘‘the technology is also expected to help people who have permanently lost their voice.”
's story and Stanley Kubrick
's associated film 2001: A Space Odyssey (film)
. In this, HAL 9000
, a computer controlling spaceship Discovery One
, bound for Jupiter, discovers a plot to deactivate it by the mission astronauts Dave Bowman and Frank Poole
through lip reading
their conversations.
In Orson Scott Card
's series (including Ender's Game
), the artificial intelligence can be spoken to while the protagonist wears a movement sensor in his jaw, enabling him to converse with the AI without making noise. (He also wears an ear implant.)
Lip reading
Lip reading, also known as lipreading or speechreading, is a technique of understanding speech by visually interpreting the movements of the lips, face and tongue with information provided by the context, language, and any residual hearing....
. It works by the computer identifying the phoneme
Phoneme
In a language or dialect, a phoneme is the smallest segmental unit of sound employed to form meaningful contrasts between utterances....
s that an individual pronounces from nonauditory sources of information about their speech movement
Speech production
Speech production is the process by which spoken words are selected to be produced, have their phonetics formulated and then finally are articulated by the motor system in the vocal apparatus...
s. These are then used to recreate the speech
Speech
Speech is the human faculty of speaking.It may also refer to:* Public speaking, the process of speaking to a group of people* Manner of articulation, how the body parts involved in making speech are manipulated...
using speech synthesis
Speech synthesis
Speech synthesis is the artificial production of human speech. A computer system used for this purpose is called a speech synthesizer, and can be implemented in software or hardware...
.
Information sources
Silent speech interface systems have been created using ultrasoundUltrasound
Ultrasound is cyclic sound pressure with a frequency greater than the upper limit of human hearing. Ultrasound is thus not separated from "normal" sound based on differences in physical properties, only the fact that humans cannot hear it. Although this limit varies from person to person, it is...
and optical camera input of tongue
Tongue
The tongue is a muscular hydrostat on the floors of the mouths of most vertebrates which manipulates food for mastication. It is the primary organ of taste , as much of the upper surface of the tongue is covered in papillae and taste buds. It is sensitive and kept moist by saliva, and is richly...
and lip
Lip
Lips are a visible body part at the mouth of humans and many animals. Lips are soft, movable, and serve as the opening for food intake and in the articulation of sound and speech...
movements. The detection of speech movements by electromyography
Electromyography
Electromyography is a technique for evaluating and recording the electrical activity produced by skeletal muscles. EMG is performed using an instrument called an electromyograph, to produce a record called an electromyogram. An electromyograph detects the electrical potential generated by muscle...
of speech articulator
muscles and the larynx
Larynx
The larynx , commonly called the voice box, is an organ in the neck of amphibians, reptiles and mammals involved in breathing, sound production, and protecting the trachea against food aspiration. It manipulates pitch and volume...
is another technique. Another source of information are the vocal tract
Vocal tract
The vocal tract is the cavity in human beings and in animals where sound that is produced at the sound source is filtered....
resonance signals that get transmitted through bone conduction
Bone conduction
Bone conduction is the conduction of sound to the inner ear through the bones of the skull.Bone conduction is the reason why a person's voice sounds different to him/her when it is recorded and played back. Because the skull conducts lower frequencies better than air, people perceive their own...
called non-audible murmurs.
They have also been created as a brain–computer interface using brain activity in the motor cortex
Motor cortex
Motor cortex is a term that describes regions of the cerebral cortex involved in the planning, control, and execution of voluntary motor functions.-Anatomy of the motor cortex :The motor cortex can be divided into four main parts:...
obtained from intracortical microelectrode
Chronic Electrode Implants
Chronic Electrode Implants are electronic devices implanted into the brain. They may record electrical impulses in the brain or they may stimulate neurons with electrical impulses from an external source.- Clinical applications and direction :...
s.
Uses
Such devices are created as aids to those unable to create the sound phonationPhonation
Phonation has slightly different meanings depending on the subfield of phonetics. Among some phoneticians, phonation is the process by which the vocal folds produce certain sounds through quasi-periodic vibration. This is the definition used among those who study laryngeal anatomy and physiology...
needed for audible speech such as after laryngectomies
Laryngectomy
Laryngectomy is the removal of the larynx and separation of the airway from the mouth, nose and esophagus. The laryngectomee breathes through an opening in the neck, a stoma. This procedure is usually performed in cases of laryngeal cancer...
. Another use is for communication when speech is masked by background noise
Background noise
In acoustics and specifically in acoustical engineering, background noise or ambient noise is any sound other than the sound being monitored. Background noise is a form of noise pollution or interference. Background noise is an important concept in setting noise regulations...
or distorted by self-contained breathing apparatus. A further practical use is where a need exists for silent communication such as when privacy is required in a public place, or hands-free data silent transmission is needed during a military
Military operation
Military operation is the coordinated military actions of a state in response to a developing situation. These actions are designed as a military plan to resolve the situation in the state's favor. Operations may be of combat or non-combat types, and are referred to by a code name for the purpose...
or security operation.
In 2002, the Japanese company NTT DoCoMo
NTT DoCoMo
is the predominant mobile phone operator in Japan. The name is officially an abbreviation of the phrase, "do communications over the mobile network", and is also from a compound word dokomo, meaning "everywhere" in Japanese. Docomo provides phone, video phone , i-mode , and mail services...
announced it had created a silent mobile phone
Mobile phone
A mobile phone is a device which can make and receive telephone calls over a radio link whilst moving around a wide geographic area. It does so by connecting to a cellular network provided by a mobile network operator...
using electromyography and imaging of lip movement. ‘‘The spur to developing such a phone,” the company said, ‘‘was ridding public places of noise,” adding that, ‘‘the technology is also expected to help people who have permanently lost their voice.”
In fiction
The decoding of silent speech using a computer played an important role in Arthur C. ClarkeArthur C. Clarke
Sir Arthur Charles Clarke, CBE, FRAS was a British science fiction author, inventor, and futurist, famous for his short stories and novels, among them 2001: A Space Odyssey, and as a host and commentator in the British television series Mysterious World. For many years, Robert A. Heinlein,...
's story and Stanley Kubrick
Stanley Kubrick
Stanley Kubrick was an American film director, writer, producer, and photographer who lived in England during most of the last four decades of his career...
's associated film 2001: A Space Odyssey (film)
2001: A Space Odyssey (film)
2001: A Space Odyssey is a 1968 epic science fiction film produced and directed by Stanley Kubrick, and co-written by Kubrick and Arthur C. Clarke, partially inspired by Clarke's short story The Sentinel...
. In this, HAL 9000
HAL 9000
HAL 9000 is the antagonist in Arthur C. Clarke's science fiction Space Odyssey saga. HAL is an artificial intelligence that interacts with the astronaut crew of the Discovery One spacecraft, usually represented as a red television-camera eye found throughout the ship...
, a computer controlling spaceship Discovery One
Discovery One
United States Spacecraft Discovery One is a fictional spacecraft appearing in The Space Odyssey series, including the movie 2001: A Space Odyssey. Discovery One is a large, nuclear-powered interplanetary spaceship.-History:...
, bound for Jupiter, discovers a plot to deactivate it by the mission astronauts Dave Bowman and Frank Poole
Frank Poole
Frank Poole is a fictional character from Arthur C. Clarke's Space Odyssey series. In Stanley Kubrick's 2001: A Space Odyssey, Poole was portrayed by Gary Lockwood....
through lip reading
Lip reading
Lip reading, also known as lipreading or speechreading, is a technique of understanding speech by visually interpreting the movements of the lips, face and tongue with information provided by the context, language, and any residual hearing....
their conversations.
In Orson Scott Card
Orson Scott Card
Orson Scott Card is an American author, critic, public speaker, essayist, columnist, and political activist. He writes in several genres, but is primarily known for his science fiction. His novel Ender's Game and its sequel Speaker for the Dead both won Hugo and Nebula Awards, making Card the...
's series (including Ender's Game
Ender's Game
Ender's Game is a science fiction novel by American author Orson Scott Card. The book originated as the short story "Ender's Game", published in the August 1977 issue of Analog Science Fiction and Fact. Elaborating on characters and plot lines depicted in the novel, Card later wrote additional...
), the artificial intelligence can be spoken to while the protagonist wears a movement sensor in his jaw, enabling him to converse with the AI without making noise. (He also wears an ear implant.)