SMARTY

 Priyanshu Agarwal
 Dipen Dave
 Ravikiran Chollangi
 Jason Lieu
What is SMARTY ?
Vision based autonomous ball sorting
robotic system
 Wireless Robot using bluetooth

communication

 Computer vision guidance

 Sorts balls based on color

Features
 One sensor system
Replaces 4 sensors bump sensor(boundary), position
sensor(bot position), ultrasonic sensor (ranging), infrared
sensor(color identification)
 Bluetooth communication
No wires No interference
 Identifies balls based on color
 Runs in a Adaptive Proportional Control
 Escorts balls to their respective home

positions
Modes of Operation

 Autonomous No human intervention

 Assistive Human input, computer control

 Manual Human input and control

System Architecture
RF
RF Remote
Remote Webcam
Webcam

NX-1000
NX-1000 MATLAB
MATLAB
(RF
(RF (Serial
(Serial Port/
Port/
Communication)
Communication) Bluetooth)
Bluetooth)

Servo
Servo Motor
Motor 1
1 Board of
Servo Motor
Servo Motor 2
2 Education Bluetooth
Bluetooth
Module
Module
Servo
Servo Motor
Motor 3
3 (Onboard)
Algorithms
Algorithms
 System overview
 Image Processing
 Calibration
 Navigation
Algorithms
 System overview
 Image Processing
 Calibration
 Navigation
System overview
Algorithms
 System overview
 Image Processing
 Calibration
 Navigation
Image Processing
 HSV (Hue, saturation, value)
color model
 RGB (Red, green, blue) color

model
HSV model
1. Read the input image and display it.
2. Convert the input RGB model to HSV model.
3. Eliminate the reflection of light from ball’s surface i.e shiny spots.
4. Extract the Hue and Saturation Component’s from the HSV Model and store them
separately.
5. Perform a gray-level slicing of the Saturation image.
6. Set the minimum and maximum Threshold values as well the Sigma used for the
Laplacian of Gaussian.
7. Apply Canny Filter to the Hue Image and find the red and yellow balls boundaries.
8. Apply Canny Filter to the Saturation Image.
9. Inverting the Output of applying the saturation image.
10. Performing Morphological Operation ‘Opening’.
11. Performing Morphological Operation ‘Closing’.
12. Find the Blue and Green Balls boundaries.
 
Step 1
Step 7
Step 8
Step 9
Step 10
Step 11
RGB model
 Original Image
RGB model
 Thresholding to remove noise
RGB model
 Separate Bot
RGB model
 Separate Red, Green and Blue channels
RGB model
 Separate objects
RGB model
 Find area and centroids
Algorithms
 System overview
 Image Processing
 Calibration
 Navigation
Calibration
 Bot calibration
Calibration
 Ball calibration
Algorithms
 System overview
 Image Processing
 Calibration
 Navigation
Navigation
Navigation
Modes of operation – How?

 Autonomous No human intervention

 Assistive Human input, computer control

 Manual Human input and control

Assistive mode
Manual Mode
Demonstration