Advanced LabVIEW Programming

Concepts for Multicore Systems

Agenda

Multithreading Overview
Parallel Programming Techniques
Multicore Programming Challenges
Application Examples
Conclusions

Multithreading Overview

Impact on Engineers and Scientists

Engineering and scientific applications are typically on
dedicated systems (i.e. little multitasking).

Creating Multithreaded Applications

Engineers and scientists must use threads to benefit
from multicore processors.

What are Processes and Threads?

Process
Every program executes in a process
Processes provide resources needed to execute
Each process has at least one thread

Thread
Entity within a process that can be executed
Shares resources of the process
Has individual thread resources

How the OS Handles Thread Scheduling

Different OSs use different thread scheduling techniques
With a multicore CPU, the OS will simultaneously execute
different threads in different processors when possible

Waiting

Ready to Run
Thread

Thread
Thread
Thread
Thread

Thread

Thread

Running
Thread

Thread

Core 1

Core 2

Thread

Example of different thread states

How LabVIEW Implements Multithreading

Implicit Parallelism / Threading
Automatic Multithreading using LabVIEW
Execution System
Parallel code paths on block diagram can execute
in unique threads

Explicit Parallelism / Threading

Timed Structures spawn a single thread

Automatic Multithreading in LabVIEW

LabVIEW automatically divides each application into multiple
execution threads (originally introduced in 1998 with LabVIEW 5.0)
Original goal of multithreading was to more elegantly handle
hardware interrupts and allow for a responsive UI

Automatic Multithreading in LabVIEW

An oversimplification of threading in LabVIEW is
shown below
Main idea is parallel code paths will execute in
unique threads

thread
thread
thread

How the LabVIEW Execution System Works

1

thread
thread
thread
thread

1. LabVIEW compiler analyzes diagram and

assigns code pieces to clumps

# of threads scales based
2. Information about which pieces of code
on # of CPUs
can run together are stored in a run queue
3. If block diagram contains enough parallelism, it will
simultaneously execute in all system threads

Multithreading Features in LabVIEW 8.5

Scale # of execution system threads based on
available cores
Improved thread scheduling for LabVIEW timed
loops (to allow multicore support)
Processor Affinity capability with Timed Structures
Real-Time Features:
Support for Real-Time targets with Symmetric Multiprocessing
Real-Time Execution Trace Toolkit 2.0

Deterministic Real-Time Systems

LabVIEW 8.5 adds Symmetric Multiprocessing
(SMP) for real-time systems.

Assigning Tasks to Specific Cores

In LabVIEW 8.5, users can assign code to specific
processor cores using the LabVIEW Timed Loop.

Explicit Threading with Timed Structures

Code within timed
structures will
execute in
precisely 1 thread
(no more)
Can be assigned a
relative priority
Can be used to set
processor affinity

DEMOS
Multithreading in LabVIEW
Implicit Parallelism (Automatic
Multithreading in LabVIEW)
Explicit Parallelism (Control of threads
with Timed Loops)

Parallel Programming Techniques

Parallel Programming in LabVIEW vs. C

LabVIEW Advantages

LabVIEW Disadvantages

Inherent Parallelism
Automatic multithreading
LabVIEW is cross-platform
(Windows, Mac, Linux), no
need to learn different
threading APIs
Support for parallel libraries
such as Intel Math Kernel
Library (MKL)

Just like C, optimization

techniques must be applied to
take full advantage of
multicore there is no silver
bullet
No support for compiler
optimizations with OpenMP
(but NI is working to add
similar capability)

Application Example: Control System for

Autonomous Vehicle
Chose LabVIEW for
multithreading approach
over C
LabVIEW running on two
quad-core HP servers

The ability of LabVIEW to automatically multithread our application, in

addition to the optimizations we performed in the language itself,
drastically reduced our development time.
- Michael Fleming, President of Torc Technologies

Multicore Programming Goals

1. Increase code execution speed (# of FLOPS)
2. Maintain rate of execution but increase data throughput
3. Evenly balance tasks across available CPUs (fair distribution of
processing load)
4. Dedicate time-critical tasks to a single CPU (Real-time use-case)

What are the Trade-offs?

Parallel programming overhead is greater than sequential
Optimizing for speed can come at a cost of more memory
utilization

Example: Unbalanced Parallel Tasks

CPU Core

Task A

Task B

CPU Core

Task E

Task F

CPU Core

CPU Core

Task C

Task D

time

Goal: Balanced Parallel Tasks

CPU Core

CPU Core

CPU Core

CPU Core

Task A

Task C

Task E

Task B

Task F

Task D

time

Application Example: Real-Time

Control
Wind Tunnel Safety-of-Flight
system at NASA Ames
Research Center
Benchmarks Results
Ran on PXI-8106 RT Controller
Time-Critical loop was reduced from
43% CPU load to 30% CPU load on one
CPU of the PXI-8106RT
A core leftover for processing of noncritical tasks.

Image Source: http://windtunnels.arc.nasa.gov

Application Decomposition
First step is to break down program into
core components

Application

Tasks

Data

Data Flow

Choosing the Right Parallel Strategy

Task Paradigm
Best suited for programs with independent functionalities
that have minimal data dependencies

Data Paradigm
Best suited for data operations that are completely
independent

Data Flow Paradigm

Best suited for data that have dependencies and require
prior computation

Parallel Strategies
Task
Paradigm

Application

Data
Paradigm

Data Flow
Paradigm

Task Parallelization
Divide & Conquer

Geometric
Recursive
Pipeline
Wave Front

Task Parallelism
Tasks
Code is comprised of logically independent blocks
of functionality

Divide & Conquer

A section of code that can be decomposed into
parallelized subsections, once completed results
are merged together

Task Parallelism
Not all code requires sequential execution
Isolate independent chunks of code and
mark them as tasks

Task Parallelism
Not all code requires sequential execution
Isolate independent chunks of code and
mark them as tasks
Task A
Task B
Task C

Divide & Conquer

Useful for problems that can be broken into
subsections
Recursive algorithms such as quick sort and
merge sort
Break the problem into manageable segments
according to your resources

Divide & Conquer

problem

split
subproblem

subproblem

split
subproblem

split
subproblem

solve

subproblem

solve

subsolution

subsolution

subproblem

solve

solve

subsolution

subsolution

merge

merge

subsolution

subsolution

merge
solution

Data Parallelism
Geometric Decomposition
No dependencies in data
Could be completely parallelized if enough
resources were available

Recursive Structure
Similar to Divide & Conquer strategy. Data is
inherently recursive and can be split up into
parallelized subsets

Data Parallelism
You can speed up processor-intensive operations
on large data sets by segmenting the data.
Data Set
CPU Core
CPU Core

Signal Processing
CPU Core
CPU Core

Result

Data Parallelism
You can speed up processor-intensive operations
on large data sets by segmenting the data.
Data Set
CPU Core

Signal Processing

CPU Core

Signal Processing

CPU Core

Signal Processing

CPU Core

Signal Processing

Combine
Results

Application Example: High-Speed Control

Max Planck Institute (Munich, Germany)
Plasma control in nuclear fusion tokamak with
LabVIEW on 8-core system using data parallelism
technique
with LabVIEW, we obtained a 5X processing speed-up on an
octal-core processor machine over a single-core processor
Dr. Louis Giannone
Lead Project Researcher
Max Planck Institute

Data Flow Parallelism

Pipelining
Data must go through a series of computations
(think of an automobile assembly line)

Wave Front
Data has dependencies but can be computed if
prior elements are computed

Pipelining
Many applications involve sequential,
multistep algorithms
Applying pipelining can increase performance
2

Acquire
1

Filter Analyze Log

3

Acquire
1

Filter Analyze Log

3

time
t0

t3

t4

t7

Pipelining Strategy
CPU Core

Acquire

Filter

CPU Core

Analyze

CPU Core

Log

CPU Core

t0

t1

t2

t3

time

Pipelining Strategy
CPU Core

Acquire

Acquire

Filter

CPU Core

Filter

Analyze

CPU Core

Analyze

Log

CPU Core

t0

t1

t2

t3

Log

time

Pipelining Strategy
CPU Core

Acquire

CPU Core

Acquire

Acquire

Acquire

Filter

Filter

Filter

Filter

Analyze

Analyze

Analyze

Analyze

Log

Log

Log

CPU Core

CPU Core

t0

t1

t2

t3

Log

time

Pipelining in LabVIEW
Sequential

Pipelined

?
Note: Queues may also be
used to pipeline data between
different loops

or

Key Considerations for Pipelining

Consider # of Processor Cores when determining # of
pipeline stages
Be sure to balance stages since longest stage will be
limiting factor in performance speed-up
Example:
Task A
Unbalanced
Pipeline
Task C
Task B

Application Example: Communications Test

AmFax Ltd. (United Kingdom)
Created wireless test systems for next-generation
phones using pipelined LabVIEW architecture based
on pipelining technique
With LabVIEW and the dualcore embedded controller we are
achieving up to 5x speed savings.
Mark Jewell
BDM Wireless
AmFax Ltd.

Tips for Balancing Pipeline Stages

Use LabVIEW Benchmarking techniques
Perform basic benchmarking with
timestamps and VI Profiler

Wave Front
Dependencies exist
in elements of the
data structure
For instance, the value
at (i,j) requires
computed value at
(i-1,j-1)

Wave Front
As long as the dependencies are satisfied
multiple operations can be carried out on
data

Wave Front
Wave Front effect appears as parallel
executions iterate over data
1

Wave Front
Practical Applications
Error Diffusion for Black & White Printers
Image Processing & Filtering

Predicting Speed Increase Amdahls Law

Theory used to calculate the expected speedup of a
parallelized implementation (Relative to serial).

Pk = % of instructions affected
Sk = Speed increase factor
K = Section of code label
N = Total number of sections
Stotal = Total speed change factor

DEMOS
Parallel Programming Techniques
Divide and Conquer/Recursion
Data Parallelism
Pipelining
Amdahls Law

Multicore Programming
Challenges

Multicore Programming Challenges

Programming Caveats
Thread Synchronization
Race Conditions
Deadlocks

Potential Code Bottlenecks

Debugging
More things happening at one time

Memory

Data transfer between processor cores and cache considerations

Thread Synchronization
With OS scheduling, there is no guarantee when
threads will execute without using synchronization
primitives
Order of events may change at each execution due
to the way the threads are scheduled
First execution

Thread
1

Thread
2

Thread
3

Second execution

Thread
2

Thread
3

Thread
1

Third execution

Thread
3

Thread
1

Thread
2

Race Conditions
This issue occurs when threads manipulate shared resources
simultaneously
Common problem when code is migrated from single CPU
system to multicore (software was not originally created for
multicore)
No synchronization utilized between threads, results in
anomalous behavior
Ex: Two threads simultaneously writing to one memory
location

Thread 1 Data

Thread 2 Data

Deadlocks and Thread Starvation

Deadlock occurs when two or more threads wait for
resources occupied by another thread in the same
group
Thread deadlock is
analogous to traffic
deadlock, nothing can
proceed

Thread Starvation occurs when a thread is

perpetually denied the resources it needs to execute

Addressing Multicore Challenges with

LabVIEW
Dataflow paradigm is a key
benefit for multicore
programming
Helps synchronize threads
and prevent race
conditions/deadlocks
thread

thread

Parallel code paths are

synchronized and order of
execution is determined by
LabVIEW wires

Synchronization in LabVIEW
When more synchronization
is required, use
synchronization mechanisms:

Notifiers
Queues
Semaphores
Rendezvous
Occurrences

Potential Bottlenecks: Shared Resources

1. Data Dependencies
Example: Data stored in global and shared
variables that need to be accessed by different VIs
would be a shared resource

2. Hard Disk
Example: Computers can only read or write to
the hard disk one item at a time (File I/O cannot
be made into parallel operations)

3. Blocking Items
Example: Non-reentrant VIs

4. Entire software stack must support multicore

Non-reentrant VIs

VIs that are non-reentrant cannot be called simultaneously,

one call runs and the other call waits for the first to finish
before running

To make a VI reentrant, select FileVI Properties, select the Execution

category and choose Reentrant execution.

Multithreaded Software Stack Support

Software Stack
Development tool Support provided on the
operating system of choice; tool
facilitates correct threading and
optimization

Example: Multithreaded
nature of LabVIEW and
structures that provide
optimization

Libraries

Thread-safe, re-entrant libraries

Example: BLAS libraries

Device drivers

Drivers designed for optimal

multithreaded performance

Example: NI-DAQmx driver

software

Operating system Operating system supports

multithreading and multitasking
and can load balance tasks

Example: Support for

Windows, Mac OS, Linux OS,
and real-time operating
systems

Application Example: Eaton Corporation

Eaton created a portable in-vehicle test system for
truck transmissions using LabVIEW.
Acquired and analyzed 16 channels on single core using
DAQmx multithreaded driver
Now acquire and analyze 80+ channels on multicore
There was no need to rewrite our
application for the new multicore
processing platforms.
Scott Sirrine
Lead Design Engineer
Eaton Truck Division

Debugging Methods
Functional Debugging
Trace Debugging
Performance Counters

Functional Debugging
LabVIEW supports debugging parallel code for
functional correctness
Use basic LabVIEW debugging tools (highlight
execution, probes, etc.) to ensure code is
functionally correct

Trace Debugging
On real-time systems, trace debugging can show thread
activity at the OS level
Thread activity on each core is displayed by selecting a
particular CPU

Performance Counters
Performance counters provide
detailed system information such as
CPU usage, memory usage, and cache
hits/misses
LabVIEW does not natively support
performance counters but can call
Windows counters programmatically
Example utilities for performance
counting include:
Windows Perfmon
Intels VTune

DEMOS
Debugging Methods
Functional Debugging
Trace Debugging
Performance Counters

Memory Considerations
Data transfer between cores
Cache considerations

Data Transfer Between Cores

Physical distance between processor and the
quality of the processor connections can have
large effect on execution speed

Cache Considerations
Multicore processors
typically utilize a shared
cache
Common cache problem is
false sharing where two
cores write to the same
cache line and cause
performance degradation
For cache optimization, use
processor affinity

Cache Optimization with Processor Affinity

Setting processor affinity tells the OS which processor to
execute the code on
Processor affinity can prevent OS from scheduling threads in a
configuration that hurts cache usage

Application Examples

Example #1 Multi-channel Acquisition

and Signal Processing
Overview: Two channels are read from a digitizer and an
FFT operation is performed on the data.

2 Channels
from a
Digitizer

Recommendation
1. Read data channels separately
2. Perform FFT operations in parallel

Example #2 Operations on Large Data Sets

Overview: Multiplication of two matrices puts heavy load on
processor, especially when the matrices are large
Multiply
Matrices
Matrix 1
Matrix 2

Result

Recommendation
Split data into subsets and then perform the operation.

Example #3: Multi-loop Producer /

Consumer Architecture with Queues
Multiloop architectures use queues to pass data between
parallel loops
Queues allow each
loop to run at an
optimal rate
In this example, the
acquisition is not
slowed by the write to
disk task

Recommendation
Balance acquisition rate and processing rate for
maximum throughput
Acquire from Scope

Dequeue
Element
Data Decomposition

Digital Output

7th Order
Low-pass filter
Enqueue
Element

DEMO
Multiloop Producer / Consumer

Conclusions
Multithreading - LabVIEW offers implicit parallelism
(automatic multithreading) and explicit parallelism
(timed structures)
Parallel Programming Techniques - There is no silver
bullet, the advantage of LabVIEW is that parallelism is
much easier expressed by the language

Conclusions (continued)
Multicore Programming Challenges - Debugging and
memory considerations have evolved with multicore
and play an important role
Application Examples - With minor modifications,
typical LabVIEW applications can be optimized for
multicore

Resources
www.ni.com/multicore

How to Develop
your LabVIEW skills

Fast-Track to Skill Development

New User

Experienced User

Courses

LabVIEW Advanced I

Core Courses
Begin
Here

LabVIEW
Basics I

Advanced User

LabVIEW
Basics II

Certifications
Certified LabVIEW
Associate Developer Exam

LabVIEW
Intermediate I

LabVIEW
Intermediate II

Certified LabVIEW
Developer Exam

If you are unsure

-Quick LabVIEW quiz
-Fundamentals exam

ni.com/training

Certified LabVIEW
Architect Exam

Certification

Training Membership the

Flexible Option
Ideal for developing your skills with NI
products
12 months to attend any NI regional or online
course and take certification exams
$4,999 for a 12 month membership in the USA

Next Steps
Visit ni.com/training
Identify your current expertise level and
desired level
Register for appropriate courses
$200 discount for attending LV Dev Day event!