Data Acquisition Reference Architecture

Developer Education Day 2011

Kitt Farrar Cell: (513) 894-2321 kitt.farrar@ni.com

Why is Software Important?

Total Measurement Application Cost =

Price of HW + Price of SW + Cost of Development

Visible Price Hidden Costs

36%, Price of HW & SW

64%, Development Time Costs

Source: National Instruments Survey of Data Acquisition Customers and Prospects 2005

Oil Well Fracture Pump Monitoring and Analysis Using NI LabVIEW

Application: A system that monitors the operating parameters of a reciprocating pump used in well servicing applications.

Challenge: Building an advanced monitoring system that can survive being mounted directly to an oil well servicing pump in a rugged environment while performing advanced analysis on sensor data.
Products: NI CompactRIO, LabVIEW, LabVIEW RealTime and LabVIEW FPGA modules, and NI Single-Board RIO Key Benefit: The high-speed I/O and analysis catch the momentary pressure spikes and vibration indications of the pumps. LabVIEW has made the software development side much quicker than our past experiences in C-based programming. What most C programmers take two years to do, we can accomplish in a couple of months. We can use that time savings to get to market quicker and capitalize on our competitors lag time. Robert Stewart, Supreme Electrical Services Inc.

Agenda
Framework Definition Framework Outline Demo - Overview Framework Modules Demo Adding Your Own Display Windows Additional Systems Engineering Components Networking Your Data Display Substitute Other Options for Logging To Disk References

Data Acquisition Reference Design

Open framework: use as-is or custom tailor Core functionality:
Configuring Data Acquisition Acquiring Data Logging Data to Disk Displaying Data

Leverages several SE software components

Four Processes

1. Handle all GUI events from the main user interface. 2. Process messages from user interaction or other external processes. Additional display windows or plug-ins can be launched from this process. 3. Acquire data continuously. Data is sent via a queue to the data logging process and is also published via a notifier for any display windows. 4. Log data.

Inter-Loop Communication
Two types of data passed between processes:
Commands Data

Commands are sent using the Asynchronous Message Communication (AMC) library Time Domain Data is sent using two methods:
Queues (lossless) Notifiers (potentially lossy)

Demonstration

10

Source Code Organization

Multiple Project Library (.lvlib) files
Protects VI namespaces Defines VI scope (i.e. public or private) Organizes the VIs

11

Configuration Module
Provides the user a way to configure the system via
Front Panel Interaction Loading saved configurations from file Manipulating configurations currently in memory

12

Data Acquisition Module

Queue-driven State Machine with 4 states
Idle Start Acquire Stop

Acquires data from the configured source:

Hardware Simulated Hardware File

13

14

Data Logging Module

Also a queue-driven state machine Saves time domain data to disk using a lossless queue buffering mechanism

15

Display Windows Module

Launches windows as reentrant VIs using VI server from the Main Message Processor Display windows are all Queued State Machines Add your own display windows using the Display Template.vit

16

17

Demonstration: Adding Your Own Display Windows

18

Additional SE Software Components

Asynchronous Message Communication
Sending commands between processes

Channel Monitor XControl

Intuitive front panel channel monitoring utility

FlexSignal Reference Library

Acquiring from real H/W, simulated H/W, or a file

GXML Reference Library

Stores system configuration information in XML format

19

Asynchronous Message Communication

Library for sending messages
Within a Process

Between Processes

20

Creating the Message Queue and Inserting Messages

21

Processing Messages

22

Networking Your Data Display

One to Many
LabVIEW Remote Front Panels or LabVIEW Web Services

One to One
Network Streams STM (Simple TCP Messaging)

23

Network Streams (LabVIEW 2010)

New API for high-throughput streaming over the network
Open/Read/Write/Close API Multi-element read/write supported

Supported on LabVIEW Real-Time and Windows Performance is comparable to TCP

24

Seamlessly Stream Data Over the Network

25

Pitfalls of Streaming with Variables

Lack of flow control can result in data loss
Data may be lost if the TCP/IP connection is dropped Data loss does not result in an error, only a warning
Machine 1
Server
5 6 7 1 2 3 4

Machine 2
Client Readers
4 5 6 7 1 2 3 4

Client Writers
1 2 3 5 6 7 4

26

Network Streams in Action

Machine 1
Writer Endpoint
9 3 2 1 8 5 4 7 6 9 8 5 4 7 6 5 4 3 2 1 Acknowledge (2) Flow Control (3) (2)

Machine 2
Reader Endpoint
5 4 3 2 1 5 4 3

27

Options for Logging to Disk

1. ASCII 2. Binary 3. TDMS

28

ASCII: Characteristics
Human readable text file Data represented as string Common for low speed DAQ

29

ASCII: Write to File

Convert to String

30

Binary: Characteristics
Not human readable Common for high speed DAQ Common for multiple channel DAQ

31

Binary: Write to File

32

TDMS: Characteristics
NIs recommended file format Binary header Binary data storage Optimized for high-speed streaming Versatile data buffers Recommended for all DAQ

33

TDMS: Write to File with Primitives

34

TDMS: Create & Set Properties

35

TDMS: Write Data & Close

36

Benchmarks: Performance Comparison

20 Chunks of 10,000 Bytes Each

37

Benchmarks: Size Comparison

20 Chunks of 10,000 Bytes Each

38

Examples
1. Writing data to an ASCII file with timestamps 2. Writing waveform data to file with timestamps 3. Using the Microsoft Report Generation Toolkit

39

For Further Reference

Data Acquisition Reference Design for LabVIEW http://zone.ni.com/devzone/cda/tut/p/id/11805 Data Acquisition Reference Application (The Code) http://zone.ni.com/devzone/cda/epd/p/id/6438 Refer to the Data Acquisition Reference Application link for hyperlinks to the additional reference software libraries that you will need.

40

Additional References
TDM Streaming (TDMS) VIs
http://zone.ni.com/devzone/cda/tut/p/id/3539

TDMS File Format Internal Structure

http://zone.ni.com/devzone/cda/tut/p/id/5696

TDMS Excel Add-In for Microsoft Excel Download

http://zone.ni.com/devzone/cda/epd/p/id/2944

Creating a Report in Microsoft Excel Using the LabVIEW Report Generation Toolkit
http://zone.ni.com/devzone/cda/tut/p/id/3178

41

Review
Framework Definition Framework Outline Demo - Overview Framework Modules Demo Adding Your Own Display Windows Additional Systems Engineering Components Networking Your Data Display Other Options for Logging To Disk References

42

Data Acquisition Reference Architecture

Developer Education Day 2011

Questions?
Kitt Farrar Cell: (513) 894-2321 kitt.farrar@ni.com