Automated Model-based Testing of a Complex Avionics Algorithm for the NH-90 helicopter

Edwin van de Sluis sluis@nlr.nl Senior Scientist Avionics Software NLR Amsterdam
Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR

NH90 NATO Frigate Helicopter (NFH) variant

Primary roles:
Anti-Submarine Warfare (ASW) Anti-Surface Unit Warfare (ASUW) Search and Rescue

Secondary roles:

Casualty and/or medical evacuation Personnel, material and troop transport

3 crew members; PILOT, TACCO, SENSO A second cabin can be installed for a 4th Replaces Lynx helicopter of Royal Netherlands Navy

NH90 program - NLR involvement

long life cycle (1980-2040)

1980 1990 2000 2010 2030

Feasibility study

Development and qualification

Testing

Operation

Wind tunnel testing

Tactical System Equipment

Stork produced NH-90 tail

Heli-Ship Qualification

Sustainment
3

NH90 Avionics with NLR involvement

Intermediate Gear Box Sensor Qualification Support

External Fuel Panel (EFP)

Cockpit Fuel Panel (CFP)

Remote Frequency Indicator (RFI)

Matching Tracks by Association and Correlation(MTAC) Landing Gear Support Nose Wheel Steering System Controller (NWSSC)
4

Location CFP, Cockpit Fuel Panel RFI, Remote Frequency Indicator

Several sensors: introduce operator workload

Data Link 11

Radar FLIR ESM

Sonics

Solution: sensor fusion with MTAC

MTAC: Matching of Tracks for Association and Correlation MTAC is a software module that is part of the NFH Mission Tactical Computer (MTC, developed by Agusta) MTAC supports tactical situation awareness compiled from real-world object observations from tactical sensors and other Participating Units Primary objective: reduction of tactical operators work load by (un-)linking tracks: (dis-)association or (de-)correlation
Note: Track = data structure containing observation of real-world object

NH90 MTAC

Tactical information sources

Tactical Database

Tactical Object Management

LOCAL TRK TN 7122 REM TRK TN 6542 REJECT ACCEPT

radar ESM FLIR sonics Data Link

Track information Vessel library

Alert

MTAC

MTAC: Matching of Tracks for Association & Correlation

8

MTAC State Vector Matching

State vector Match Quality (MQ) is computed from distance between track positions distance between track position and bearing speed and course difference

r v1
p1

Bearing

r v2

x2 , y 2
p 2

r v1

x1 , y1

origin Uncertainty area (from track quality) Uncertainty area (from bearing accuracy)

MTAC Attribute Matching

Attribute Match Quality (MQ) wordt verkregen uit Basic Identity (friend, neutral, suspect, hostile, ) Type (frigate, aircraft carrier, ) Classification (subsurface only, probably sub, certainly sub, acoustic positive, ) IFF mode 2 response Fleet size (single, few, many) Propulsion (nuclear, not nuclear, ) Depth Radar cross-section Acoustic footprint Radar signature

10

MTAC characteristics
Implemented in Ada, integrated with MTC software, real-time Qualified to NH90 SDHB level 2 (~ DO-178B level C/D) Real-Time matching of state vector and attributes of two tracks (observations) from Tactical DataBase
Time_gate Attr_gate

Complex algorithm: ca. 50 formulas 64 pages specification No. of inputs: ca. 30 per track No. of output: 1; Match Quality
( 0 < MQ <= 10)
m.c. track ref. track

Position gating (4.2.2/4.2.3) Speed gating (4.2.2/4.2.3) Course gating (4.2.2/4.2.3) Position matching (6.1/6.2) Attribute gating (4.1/An.N) Time gating (4.2.1) Time alignment (3.1/3.2) Attr_gate Speed matching (6.1/6.2) Course matching (6.1/6.2) SV element availability (6.1/6.2) Category matching (5.3.1/An.N) Basic Ident. matching (5.3.2/An.N)

Position_gate or Speed_gate Match_gate

Cat_alert BI_alert Class_alert or Attr_alert

MQ_attr_low or Attr_low ML_attr_low

Course_gate

MQ_p ML_p,acc MQ_v ML_v,acc

a e b f Attr_alert or MQ_sv (6.3) ML_low 5 (2.2/6.4) Attr_low Match_gate

Time_gate

MQ_

c ML_sv (6.3) 5 (2.2/6.4) Clip to alert level (An.N) 1/0 MQ

ML_available

Cat_alert 1/weights MQ_id ML_id BI_alert MQ_class Wclass Wclass Wid Wid MQ_attr_prim 0.8 (5.4.1/5.5.1) (5.5) MQ_attr_Th ML_attr_prim (5.4.1/5.5.1) 0.8 (5.5) (5.4) MQ_attr_low

Classification matching (5.3.3/An.N)

ML_class Class_alert MQ_idamp ML_idamp MQ_iff ML_iff MQ_size ML_size MQ_prop ML_prop MQ_depth ML_depth MQ_rs ML_rs MQ_af ML_af MQ_re ML_re

1/weights 1/weights Widamp Widamp Wiff Wiff Wsize Wsize Wprop Wprop Wdepth Wdepth Wrs Wrs Waf Waf Wre Wre 1/weights ML_attr_sec (5.4.1/5.5.1) MQ_attr_sec (5.4.1/5.5.1)

Ident. Amp. matching (5.3.4) IFF mode 2 matching (5.3.5) Size matching (5.3.6) Prop. Type matching (5.3.7)

MQ_attr

ML_Th 5 (2.2/6.4) ML_attr_Th ML (2.1) ML_low

0.2 (5.5) ML_attr

(5.4)

ML_attr_low

5 (2.2/6.4)

=> Verification challenge!

Depth matching (5.3.8) Xref ISAR matching (5.3.9) Xref acous. matching (5.3.9) Xref emitter matching (5.3.10)

0.2 (5.5)

11

Verification solution: MTAC model

Separate, dissimilar model of algorithm implemented in Matlab Input: Excel CSV files (command-line i/f), or GUI based

12

Model-Based Test Approach

AdaTEST (IPL) test harness

CSV 2 Ada

Test Input

MTAC Actual Test Results Ada Code

? =

Test Report
Expected Test Results

Test inputs
(CSV files)

MTAC MatLab Model

Expected Test Results

13

Automation
UNIX scripts to automate test case compiling and execution, including test oracle consultancy Result: (regression) test pass/fail But also support for detailed result inspection Example Test: Position matching (ca. 172 cases) Course matching (ca. 80 cases) Speed matching (ca. 30 cases) Point Track Quality (all 49 combinations) Basic Identity matching (all 81 combinations) IFF mode 2 matching (>200 combinations) Identity Amplification matching (all 269 combinations) Size matching (all 25 combinations) MQ range coverage (output equivalence classes)

Match Quality Results

100000 90000 80000 70000 60000 MQ (0-100) 50000 40000 30000 20000 10000 0 1 101 201 301 401 501 TC 601 701 801 901

Total: ca. 9000 test cases

14

Concluding remarks
Successful model/oracle-based test approach for algorithm verification Supports software qualification: requirements and (statement) structural coverage MTAC has undergone several successful qualifications Support for validation still TBD NH90 mission system not fully operational before 2013 (?), so not operational validation feedback until then

15

Questions ?

16

17