ANALYSIS: Rockwell Collins readies

voice recognition for commercial

cockpits
By: Stephen Trimble
Source: Flightglobal.com
15:03 13 Oct 2014

Speech recognition technology is pervasive everywhere, it seems, except in the

commercial cockpit. Voice commands now field customer service queries, dictate text
messages and assist drivers. Cockpit avionics, however, are commanded exclusively by
human fingers and feet, which interface with the computers and controls by tapping
pedals, turning knobs, pushing buttons, touching screens and manipulating sticks or
yokes.

That is finally about to change, according to the staff at the Rockwell Collins advanced
technology centre.

Building upon decades of research and experience, the Cedar Rapids, Iowa-based
avionics specialist is poised to unveil at the NBAA convention a plan to integrate
speech recognition into the commercial cockpit.

For commercial aircraft, adding an audible interface alternative to human hands and feet
has had many false starts. Collins itself first considered introducing voice commands in
Pro Line 21, the flightdeck system deployed in the late 1990s. As Pro Line Fusion was
being developed a decade later, Collins’s marketers again touted voice recognition as a
possible feature. The technology, alas, was still not quite ready.

Meanwhile, speech commands have flourished in the most advanced combat fighters.
The Eurofighter Typhoon’s direct voice input system allows the pilot to, for example,
audibly create a waypoint at a place of interest, one of more than 90 possible voice
commands. Lockheed Martin has designed the F-35 cockpit with a system that allows a
pilot to use voice commands for manually intensive “housekeeping chores”, such as
changing a radio frequency.
 F-35 cockpit is designed to allow voice command technology for simple tasks that are
manually intensive like changing radio frequencies

Lockheed Martin

Armed with a decade of experience in combat aircraft, Collins appears poised to take
the technology into the commercial market for the first time with the next generation of
integrated avionics systems.

Perhaps aware of scepticism earned from a long history of false promises, Collins points
to two fundamental breakthroughs that suggest this time may be different.

The first development is a new leap in the accuracy of speech recognition software. The
second breakthrough is a voice command interface model that might satisfy the
concerns of regulatory agencies.

“The reason voice isn’t there today is because the accuracy of voice over a wide range
of pilots is not sufficient that it justifies that additional functionality,” says John
Borghese, vice-president of Collins’s advanced technology centre.

Several years ago, the company set a goal for voice recognition accuracy. The software
must correctly interpret well over 90% of words spoken in English, regardless of a
pilot’s accent or diction, Borghese says. The software must work as well whether the
command is spoken by a European, Asian, African or American, as well as whether it is
in a wide range of regional accents within those groups.

Collins developed speech recognition software and spent several years in testing,
canvassing the company’s wide network of pilot advisory groups. The pilots were
invited to test the system by speaking commands in cockpit simulators, with any errors
of interpretation analysed and corrected in the software algorithm. The process was
repeated until the software proved it was accurate beyond the manufacturer’s stated
threshold.

“We don’t see a difference in accuracy across accents, and that’s because we do a lot of
homework,” says senior systems engineer Geoff Shapiro.

Refining the recognition software was only part of the challenge. To field the
technology, the avionics company also needs to satisfy regulators that errors in
interpretation or execution could never jeopardise safety.

Voice recognition systems can be used to bypass long navigation menus, speeding up
data access

Rockwell Collins

On a recent visit to the advanced technology centre in Cedar Rapids, Shapiro

demonstrated how the system works in a cockpit simulator.

“This button here we’re going to call our push control voice button,” he says, wearing a
headset with his thumb over a red button on the control stick.

As long as Shapiro holds his thumb on the button, the system is “listening” to him
speak. If he lifts his thumb off the button, the speech recognition system stops listening.
“I’m going to give it a command of a heading,” Shapiro says, toggling the red activation
button.

“Turn left, heading 060,” he says.

The system audibly repeats the command and asks him to approve the heading change.
Shapiro confirms by clicking the button on his stick again. In the simulated
environment, the flight management system had been following a flight plan. Shapiro’s
voice command and confirmation suspends the flight plan and switches the autopilot to
a heading mode, then turns the aircraft left to the new heading.

Such an action is called a “macro command”, because it directs more than a single task.
Shapiro next attempts to set a new altitude, but intentionally mumbles the command.
The speech recognition system audibly requests him to “say again”. Shapiro then says,
clearly, “set altitude to 10,000”. Again, nothing happens.

Shapiro explains it’s because the automation system recognises that is not a valid
command. He needs to use a macro command that will perform three discrete tasks –
switching from heading mode to a vertical speed mode, an ascent or a descent rate and a
new altitude level.

“I can say: ‘climb and maintain flight level 310,’” he says. “It’s a macro command. It
changed my mode to vertical speed, set my bug to 31,000ft and commanded a
1,500ft/min [7.62m/s] climb.”

So far, Shapiro has demonstrated a series of voice commands that appear to effectively
control the simulated aircraft, but do not seem obviously superior to the conventional
interface – the flight director mounted on top of the glare shield.

Shapiro concedes that some voice commands do not offer time or simplicity advantages
over a manual system.

“Some things will, some things won’t,” he says. “If you’re flying along and ATC gives
you a fix that you don’t know how to spell, how do you enter that into your [centre
display unit]? Fifteen seconds later, you sheepishly call ATC, saying: ‘Hey, can you
spell this fix for us?’ It happens quite a bit. With speech recognition, you don’t have
that problem because everything is phonetic.”

Another possible advantage of a speech recognition system is helping pilots navigate

the lengthy menus of features. Aircraft cockpit systems have become increasingly
automated and sophisticated. Browsing through checklists and functions can take
dozens of seconds. So Collins and other avionics companies have been trying to
simplify the process of accessing the right information, and speech recognition is one of
the tools.

“Let’s just say I want to bring up a…,” says Shapiro, then he remembers to toggle the
voice command button.

“Bring up the engines,” he commands, then approves the confirmation request. A

display showing the status of each engine flashes on the screen.
“I can control all the flightdeck content through speech as well. We have definitely seen
serious time changes because of that,” Shapiro says. “I can get to the data quicker and
bypassing several systems. That’s why this bypassing the menu makes sense here. Not
everything will be better with voice but we believe enough will be better for this to be a
benefit.”

The introduction of voice recognition technologies adds capability, but also raises a
question: Is English always the best language to use in the cockpit? English is the
universal language of air traffic control, but it is also a workload issue for pilots with a
different primary language.

“When you fly, you speak English. It’s the standard,” Shapiro says. “Should they
maintain English speaking? Certainly, there are arguments for both. If you’re Chinese,
and you interface with your avionics using your natural language that’s one less barrier
for you. It’s a valid paradigm to look at. I don’t think we’ve made up our mind what the
solution is but the technology enables both.”