Audio Quality

Audio quality can be measured either subjectively, with human subjects who compare
audio stimuli in listening tests and generate opinion scores, or objectively, with
computations performed on the audio stimuli being tested. In both cases, the goal is to
obtain a number that reliably represents the perceived audio quality.

CRC-SEAQ (System for the Evaluation of Audio Quality) is a Windows

95/98/NT4/2000/XP software package for the subjective and/or objective evaluation of
the quality of audio signals, including speech. With its intuitive graphical user interfaces,
CRC-SEAQ provides an environment and a comprehensive set of tools for users who wish
to accurately measure audio quality while developing or evaluating audio technologies,
products and equipment. The system was developed by researchers at the
Communications Research Centre, in Ottawa, Canada who have been involved for many
years in the development of subjective and objective methods for audio quality
measurements.

CRC-SEAQ can be configured to include a Subjective Test Module and/or an

Objective Test Module. CRC-TimeSync, a companion program for time synchronizing
audio files, can also be included as a module in CRC-SEAQ or licensed separately.

Download CRC-SEAQ brochure | PDF 804 kb

You will need Adobe Acrobat to view this brochure. If this brochure is not accessible to you, please contact
louis.thibault@crc.ca for assistance.

Subjective Test Module

Current Version: v2.03.5

The Subjective Test Module is an audio playback system with a series of control
buttons and features (triple-stimulus A-B-C or multi-stimulus presentation, switching,
zooming, looping and scoring) provided in three easy-to-use graphical user interfaces
(GUI). The triple-stimulus GUI is designed to conduct ITU-R Recommendation BS.1116
compliant subjective tests and is particularly well suited for detecting and rating signals
with small audible impairments. The multi-stimulus GUI comes in two versions: a multi-
button and a multi-slider. It is ideal when signals (up to 12) with medium and large
impairments need to be compared and rated reliably against a reference, such as in the
MUSHRA test method defined in ITU-R Recommendation BS. 1534. The audio sequences
to be compared are read from files and played via any audio card with Windows MME
support. The Subjective test Module supports a variety of audio signals and file formats,
sampling frequencies and resolutions.

The Subjective Test Module is available in a 2-channel version, for mono and stereo
signals, or a multichannel version which supports up to 16 audio channels.

The Subjective Test Module can be used for:

• Formal and informal listening tests

 • Evaluating different implementations of audio processing equipment such as audio
and speech codecs, 3D audio systems, etc.
• Status or reality checks during audio system development
• Perception and psychoacoustics research
• Demonstrations involving the comparison of different audio stimuli

Key Features:

• Three intuitive graphical user interfaces (user selectable)

• Compliant with ITU-R Recommendations for testing small, medium and large
audible impairments.
• Seamless switching between signals during playback
• Versatile zooming on any portion of the audio sequences under test
• Automatic repeat (looping) of audio sequences
• Audio sequences easily grouped into listening sessions
• Built-in scoring window. Test scores readily imported into spreadsheets or other
analysis software
• Standard and user-definable rating scales
• Compatible with virtually any audio card that provides Windows MME support
• Mono, 2-channel and multichannel audio sequences
• Supports 8, 16, 20, 24, and 32 bits/sample, and normalized floating point
• Supports sampling rates of 8, 11.025, 22.05, 32, 44.1, 48, 88.2 and 96 kHz
• Supports Intel and Motorola byte ordering for audio samples
• Compatible with Microsoft WAV and headerless PCM files

Objective Test Module

Current Version: v1.5

The Objective Test Module is an "electronic ear" which performs an objective

measurement of perceived audio quality using a computerized model of the human ear.
This model, known as PEAQ, was developed jointly by experts from eight leading
research laboratories in the world, including the CRC, and is an international standard
(ITU-R Recommendation BS.1387). The model processes the two audio signals to be
compared (namely the original reference signal and an impaired version to be evaluated)
and calculates instantaneously a quality rating similar to the mean rating that would be
obtained if a formal subjective test were conducted. The basic (FFT-based) version of the
ITU-R PEAQ model is implemented in the Objective Test Module.

The measuring instrument was calibrated using data from eight subjective listening tests
conducted according to ITU-R Recommendation BS.1116. Much of the audio data in those
tests were obtained from digital audio codecs. As a result, the measurement method is
considered most reliable for evaluating codec quality. Other types of distortions may
require re-calibration of the instrument.

The Objective Test Module is useful for:

• Evaluating different implementations of audio processing equipment

• Testing equipment or circuits before they are put into service
 • Automated monitoring of the quality of an audio transmission in service
• Identifying the type and implementation of a particular audio codec
• Characterizing the performance of a coding algorithm during development
• Optimizing the cost and performance of a transmission network under given
constraints

The Objective Test Module has three modes of operation:

• On-line Mode: This mode is used for automated real-time audio quality
monitoring. The software monitors an incoming audio signal in order to recognize
and capture a pre-defined reference test signal. Once captured, the impaired test
signal is automatically synchronized with the unimpaired version stored on hard
disk, and an objective quality measure is done.
• Off-line Mode: The two signals to be compared (i.e. original reference signal and
impaired version) are stored as files on the user's PC. Both files can be
automatically synchronized before an objective quality measure is calculated.
• Play and Measure Mode: This mode simplifies the task of measuring the quality
of external audio devices. A reference signal is looped through the device under
test and recorded back onto a PC. Once the recording is stopped the software
automatically synchronizes the impaired signal with the reference signal before
performing an audio quality assessment.

Key features:

• Three modes of operation: On-line, Off-line, Play and Measure

• Audio files may be specified interactively or via a batch file
• Analysis window duration and overlap is user-selectable
• Up to 12 model output variables may be displayed
• Interactive zoom and measurement of any portion of the audio sequences under
test
• Automatic temporal alignment of audio sequences under test
• Audio files playable via the system audio card
• Supports mono and 2-channel audio sequences
• Supports 16, 20, 24, and 32 bits/sample, and normalized floating point
• Supports Intel and Motorola byte ordering for audio samples
• Supports sampling rates of 44.1 and 48 KHz
• Compatible with Microsoft WAV and headerless PCM files
• DOS command line interaction also included
Observe the variations in objective quality over time.