Sei sulla pagina 1di 24
Computer Vision and Video Processing Using MATLAB 이웅재 차장 , Senior Application Engineer 1

Computer Vision and Video Processing Using MATLAB

이웅재 차장, Senior Application Engineer

Computer Vision and Video Processing Using MATLAB 이웅재 차장 , Senior Application Engineer 1
Computer Vision and Video Processing Using MATLAB 이웅재 차장 , Senior Application Engineer 1

1

Agenda  Introduction  Video and Image Processing Blockset  Demo: optical flow  Demo:

Agenda

Introduction

Video and Image Processing Blockset

Demo: optical flow

Demo: stereo vision

Questions

Introduction  Video and Image Processing Blockset  Demo: optical flow  Demo: stereo vision 

2

Example Computer Vision and Video Processing Tasks  Detect and track objects  Count objects

Example Computer Vision and Video Processing Tasks

Detect and track objects

Count objects in a scene

Stabilize camera motion

Deinterlace video frames

Create mosaicks and panoramas

Generate depth maps from stereo image pairs

motion  Deinterlace video frames  Create mosaicks and panoramas  Generate depth maps from stereo
motion  Deinterlace video frames  Create mosaicks and panoramas  Generate depth maps from stereo

3

Common Challenges  Accessing and analyzing video data  Exploring algorithms and what-if scenarios 

Common Challenges

Accessing and analyzing video data

Exploring algorithms and what-if

scenarios

Understanding the system-wide context

Rebuilding standard algorithms

Visualizing intermediate results

Testing and validating under real-world conditions

standard algorithms  Visualizing intermediate results  Testing and validating under real-world conditions 4 4
standard algorithms  Visualizing intermediate results  Testing and validating under real-world conditions 4 4
standard algorithms  Visualizing intermediate results  Testing and validating under real-world conditions 4 4

4

4

Video and Image Processing Blockset Design and simulate video and image processing systems  Multimedia

Video and Image Processing Blockset

Design and simulate video and image processing

systems

Multimedia file I/O

Video display

Text and graphic overlays

Pre- and post-processing

Motion-based processing

Object detection and tracking

Feature recognition

Computer vision

 Motion-based processing  Object detection and tracking  Feature recognition  Computer vision 5 5
 Motion-based processing  Object detection and tracking  Feature recognition  Computer vision 5 5

5 5

Demo: Video Processing in MATLAB  Use optical flow on a video sequence 6 6

Demo: Video Processing in MATLAB

Use optical flow on a video sequence

Demo: Video Processing in MATLAB  Use optical flow on a video sequence 6 6
Demo: Video Processing in MATLAB  Use optical flow on a video sequence 6 6
Demo: Video Processing in MATLAB  Use optical flow on a video sequence 6 6

6

6

System Objects  Definition  MATLAB objects that represent time-based and data-driven algorithms, sources, and

System Objects

Definition

MATLAB objects that represent time-based and data-driven algorithms, sources, and sinks

Instantiate and configure

hReader = video.MultimediaFileReader(‘viptraffic.avi’)

Execute within a loop

step(hReader);

Other methods available

reset

isDone

close

For related information, see the following webinar:

“Object- Oriented Programming in MATLAB”

 close  For related information, see the following webinar: “Object - Oriented Programming in MATLAB”

7

7

Batch Processing Load the entire video file and process it all at once Source MATLAB

Batch Processing

Load the entire video file and process it all at once

Source

MATLAB Memory Batch Video Algorithm
MATLAB
Memory
Batch
Video
Algorithm
Processing Load the entire video file and process it all at once Source MATLAB Memory Batch

8

8 8

Stream Processing Load a video frame and process it before moving on to the next

Stream Processing

Load a video frame and process it before moving on to the

next frame

Stream

Source

MATLAB Memory Stream Processing
MATLAB
Memory
Stream
Processing
a video frame and process it before moving on to the next frame Stream Source MATLAB

9

9

Stream Processing in MATLAB is Hard myVid = mmreader (‘myvideofile.avi’); numFrames = myVid.NumberOfFrames;

Stream Processing in MATLAB is Hard

myVid = mmreader(‘myvideofile.avi’);

numFrames = myVid.NumberOfFrames;

currentFrame = read(myVid,1); numIter = 10; opticalFlowOutput = zeros([size(currentFrame) numFrames]); i = 1;

= zeros([size(currentFrame) numFrames]); i = 1; while i <= numFrames prevFrame = currentFrame;
= zeros([size(currentFrame) numFrames]); i = 1; while i <= numFrames prevFrame = currentFrame;
= zeros([size(currentFrame) numFrames]); i = 1; while i <= numFrames prevFrame = currentFrame;
= zeros([size(currentFrame) numFrames]); i = 1; while i <= numFrames prevFrame = currentFrame;
= zeros([size(currentFrame) numFrames]); i = 1; while i <= numFrames prevFrame = currentFrame;
= zeros([size(currentFrame) numFrames]); i = 1; while i <= numFrames prevFrame = currentFrame;
= zeros([size(currentFrame) numFrames]); i = 1; while i <= numFrames prevFrame = currentFrame;

while i <= numFrames

prevFrame = currentFrame; currentFrame = read(myVid,i);

Explicit state management
Explicit state management
currentFrame = read(myVid,i); Explicit state management flow = opticalFlow( currentFrame,prevFrame,‘horn -schunck

flow = opticalFlow(currentFrame,prevFrame,‘horn-schunck’, numIter,‘magitude-squared’);

-schunck ’, numIter,‘magitude - squared’ ); opticalFlowOutput(:,:,2:end) =
-schunck ’, numIter,‘magitude - squared’ ); opticalFlowOutput(:,:,2:end) =
-schunck ’, numIter,‘magitude - squared’ ); opticalFlowOutput(:,:,2:end) =

opticalFlowOutput(:,:,2:end) = opticalFlowOutput(:,:,1:end-1); opticalFlowOutput(:,:,1) = flow;

i = i+1;

Explicit indexing
Explicit indexing

end

opticalFlowOutput(:,:,1) = flow; i = i+1; Explicit indexing end implay(opticalFlowOutput) Need to maintain buffer

implay(opticalFlowOutput)

Need to maintain buffer
Need to maintain buffer
= flow; i = i+1; Explicit indexing end implay(opticalFlowOutput) Need to maintain buffer 10

10

System Objects Make it Easier reader = videoMultimediaFileReader reader.Filename = ‘myvideofile.avi’; viewer =

System Objects Make it Easier

System Objects Make it Easier reader = videoMultimediaFileReader reader.Filename = ‘myvideofile.avi’; viewer =

reader = videoMultimediaFileReader reader.Filename = ‘myvideofile.avi’; viewer = video.VideoPlayer

Initialize objects
Initialize objects

optical = video.OpticalFlow optical.Method = ‘horn-schunck’;

optical.OutputValue = ‘Magitude-squared’;

optical.ReferenceFrameDelay = 3; optical.MaximumIterationCount = 10;

while ~isDone(reader)

currentFrame = step(reader);

OF = step(optical, currentFrame); step(viewer, OF);

end

“In-the-loop” code is much simpler Implicit states, buffering, and indexing
“In-the-loop” code is much simpler
Implicit states, buffering, and indexing
Video player runs in-the-loop
Video player runs in-the-loop
“In-the-loop” code is much simpler Implicit states, buffering, and indexing Video player runs in-the-loop 11
“In-the-loop” code is much simpler Implicit states, buffering, and indexing Video player runs in-the-loop 11

11

Demo: Video Processing in MATLAB Use optical flow to detect and counting moving vehicles on

Demo: Video Processing in MATLAB

Use optical flow to detect and counting moving vehicles on a road

Demo: Video Processing in MATLAB Use optical flow to detect and counting moving vehicles on a
Demo: Video Processing in MATLAB Use optical flow to detect and counting moving vehicles on a
Demo: Video Processing in MATLAB Use optical flow to detect and counting moving vehicles on a
Demo: Video Processing in MATLAB Use optical flow to detect and counting moving vehicles on a
Demo: Video Processing in MATLAB Use optical flow to detect and counting moving vehicles on a

12 12

Video and Image Processing System Objects AlphaBlender Deinterlacer DemosaicInterpolator DeployableVideoPlayer

Video and Image Processing System Objects

AlphaBlender

Deinterlacer DemosaicInterpolator DeployableVideoPlayer EdgeDetector

IFFT2D

MorphologicalErode

Autocorrelator2D

ImageComplementer

MorphologicalOpen

Autothresholder BinaryFileReader BinaryFileWriter BlobAnalysis BlockMatcher BoundaryTracer ChromaResampler ColorSpaceConverter ConnectedComponentLabeler ContrastAdjuster

ImageDataTypeConverter

MorphologicalTopHat

ImageFilter

MultimediaFileReader

FFT2D

ImagePadder

MultimediaFileWriter

GammaCorrector

LocalMaximaFinder

OpticalFlow

GeometricRotator

MarkerInserter

PSNR

GeometricScaler

Maximum

Pyramid

GeometricTransformer

Mean

ShapeInserter

GeometricTransformEstimator Median

StandardDeviation

GeometricTranslator

MedianFilter2D

TemplateMatcher

Histogram2D

MedianFilter2D

TextInserter

Convolver2D

HistogramEqualizer

Minimum

Variance

CornerDetector

HoughLines

MorphologicalBottomHat

VideoPlayer

Crosscorrelator2D

DCT2D

HoughTransform

IDCT2D

VideoPlayer Crosscorrelator2D DCT2D HoughTransform IDCT2D MorphologicalClose MorphologicalDilate 13

MorphologicalClose

MorphologicalDilate

13

Computer Vision Example 1: Mosaicking and Stabilization  Register neighboring frames to create a large

Computer Vision Example 1:

Mosaicking and Stabilization

Register neighboring frames to create a large view of the scene or stabilize camera motion

Applications:

Security

License plate recognition

Aerial surveying

Medical imaging

motion  Applications:  Security  License plate recognition  Aerial surveying  Medical imaging 14
motion  Applications:  Security  License plate recognition  Aerial surveying  Medical imaging 14
motion  Applications:  Security  License plate recognition  Aerial surveying  Medical imaging 14

14 14

Computer Vision Example 2: Object Detection and Tracking  Detect, classify, and track objects in

Computer Vision Example 2:

Object Detection and Tracking

Detect, classify, and track objects in a scene

Tracking  Detect, classify, and track objects in a scene  Applications:  Traffic monitoring 
Tracking  Detect, classify, and track objects in a scene  Applications:  Traffic monitoring 

Applications:

Traffic monitoring

Cell counting

Lane departure warning system

objects in a scene  Applications:  Traffic monitoring  Cell counting  Lane departure warning

15 15

Computer Vision Example 3: Stereo Vision  Extract 3D information from a pair of stereo

Computer Vision Example 3:

Stereo Vision

Extract 3D information from a pair of stereo images

Applications:

Obstacle avoidance

Automotive safety

Face recognition

Scene reconstruction

Part pickers

 Obstacle avoidance  Automotive safety  Face recognition  Scene reconstruction  Part pickers 16
 Obstacle avoidance  Automotive safety  Face recognition  Scene reconstruction  Part pickers 16

16 16

Stereo Vision Basics Left Right 17 17

Stereo Vision Basics

Stereo Vision Basics Left Right 17 17

Left

Left
Stereo Vision Basics Left Right 17 17

Right

Right

17 17

Stereo Vision Algorithm  Stereo rectification  Align images horizontally  Block matching  Find

Stereo Vision Algorithm

Stereo rectification

Align images horizontally

 Stereo rectification  Align images horizontally  Block matching  Find disparity map 

Block matching

Find disparity map

Backprojection

Calculate 3D points

Align images horizontally  Block matching  Find disparity map  Backprojection  Calculate 3D points
Align images horizontally  Block matching  Find disparity map  Backprojection  Calculate 3D points
Align images horizontally  Block matching  Find disparity map  Backprojection  Calculate 3D points

18 18

Video and Image Processing Blockset Advanced Capabilities  Graphical design in Simulink  Fixed point

Video and Image Processing Blockset Advanced Capabilities

Graphical design in Simulink

Fixed point modeling

Code generation

Targeting and verification

 Code generation  Targeting and verification  DSPs  FPGAs  For more information, see

DSPs

FPGAs

For more information, see the following webinar:

“Image and Video Processing with DSPs and FPGAs”

 FPGAs  For more information, see the following webinar: “Image and Video Processing with DSPs

19

Image Acquisition Toolbox  Live video and image acquisition directly into MATLAB and Simulink 

Image Acquisition Toolbox

Live video and image acquisition directly into MATLAB and Simulink

Device property configuration

Live video previewing

GUI or functional interface

Support for multiple hardware vendors

configuration  Live video previewing  GUI or functional interface  Support for multiple hardware vendors
configuration  Live video previewing  GUI or functional interface  Support for multiple hardware vendors

20

Why Use MATLAB for Computer Vision and Video Processing?  System objects simplify coding and

Why Use MATLAB for Computer Vision and Video Processing?

System objects simplify coding and reduce errors

Algorithms for computer vision and video processing

Flexible environment enables algorithm exploration

Support for reading and writing many video file formats

Text and graphics annotations on video data

Support for cameras and frame grabbers

Embedded hardware design considerations

annotations on video data  Support for cameras and frame grabbers  Embedded hardware design considerations

21

For More Information mathworks.com/products/viprocessing/demos.html  Other examples:  People tracking  Video

For More Information

mathworks.com/products/viprocessing/demos.html

Information mathworks.com/products/viprocessing/demos.html  Other examples:  People tracking  Video

Other examples:

People tracking

Video mosaicking

Video stabilization

Lane departure warning system

Abandoned object detection

 Video mosaicking  Video stabilization  Lane departure warning system  Abandoned object detection 22
 Video mosaicking  Video stabilization  Lane departure warning system  Abandoned object detection 22

22

For More Information  Experiment with product by downloading a trial 

For More Information

Experiment with product by downloading a trial

Related demos and webinars

More detailed product information

Links to product documentation

User stories

Contact us

Sales representative

product information  Links to product documentation  User stories  Contact us  Sales representative
product information  Links to product documentation  User stories  Contact us  Sales representative

23

Questions? 24

Questions?

Questions? 24

24