Artificial Neural

Network Theory
and Application
Ashish Venugopal
Sriram Gollapalli
Ulas Bardak
Presentation Overview

 Artificial Neural Networks

 NASA profiles as a Pattern
Classification Task
 Implementing Pattern Classification
on the Silicon Recognition Neuron
Board
ANN General Overview

 Inspired by biological neuron models

 Decision are distributed throughout
system
 Components interacts at several
stages
 Components work in unison to solve
one problem
 ANN well suited to problems that don’t
have closed form algorithmic solution
 Black box model – hard to interpret
ANNs Applied

 Driving a car. Task involves

determining the direction the road is
curving based on an image
 Image pixels are input
 Curve angle is output
 Learned non-linear relationship
between pixels and roads
• hard to understand the model
Theory

 Input layer, Hidden layer, Output

Layer
 Input – one “neuron” for each pixel
 Hidden – set of neurons to store what
was learned.
 Output layer – one neuron for each of
5 turn directions – ie straight, slight
left, hard right..
 But what is a neuron?
Neurons

 Math terms: Binary decision makers

 English terms: take in some input, decide
whether is “high” or “low”, and pass this
value on.
 Similar to neurons in the brain propagating
signals.
How is the model learned?

 Once you have training examples

 Pair (feature set, classification)
 Run the back propagation algorithms
 Learns connection weights between
neurons, as well as within neuron
transfer weights.
 Network topology changes affects
learning behavior!
Unsupervised Networks

 So far we have used labeled training

examples to learn relationship
between features and response
variable.
 What if we are just exploring patterns
that exist in the features?
 Unsupervised learning / clustering…
Unsupervised Goal

 Find prototypical points within data set that

can be used to approximate distribution of
the whole data set.
 Compression / Visualization
 Typical methods
 Define a measure of similarity between data
points
 Select initial prototypes
 Move prototypes to best fit the data –
minimize error
Kohonen Net

 Unsupervised technique

 Each output neuron represents a

prototype point, input layer is
presented with data
Self Organizing Map

 Each prototype is pulled towards that

data, and brings its closest prototypes
a little closer as well…
 SOM Demo
How is it relevant to our
problem?
 We will go through:
 Definition of what we are facing
 How we can use neural nets
 How we can improve on this method
LIDAR in Space

 Data from LITE project

 Send a laser beam down from a
shuttle and get reflection readings.
 Data gathered for 53 hours in 1994.
 40GB of data
Input
Problem

 There is too much data to send and

process
 Each pixel is a number to be
transmitted.
 Each reading (one column) contains
3000 pixels.
 Need a way to transmit the
information in a more compact
fashion.
Applying Neural Nets

 Some readings are very similar to

each other.
 Define classes that will contain
multiple readings.
 Define a representative for the class
which is close enough to all the class
members.
Applying Neural Nets (cont.)

 In order to get class

 Train Kohonen Nets on data for
specific number of classes.
 Once trained, just pass in a new
reading (3000 data points) and get the
class id. Just transfer the class id.
 Go from 3000x3000 numbers to be
transferred to 3000 numbers
Example
Result – 8 classes
Result – 16 Classes
Result – 64 Classes
Result – 128 Classes
Result – 256 Classes
Problems

 Takes a while to train

 256 classes took 44 hours on a dual
P3 1GHz computer with 1.5GB of
RAM.
 Not optimal
 Classifying by the complete reading is
wasteful.
• Need one number for each reading
 Improving NN
sky
Improving NN (cont.)

 Easier version:
 Look at blocks of 50x50 pixels instead
of just 1 pixel.
 Classify as sky, etc.
 Can preprocess using kNN, basic
filters to get rid of noise in data.
 Classes can be formed from
sequence of 50 pixel blocks.
Improving NN (cont.)

 Harder version:
 Use variable sized mesh
• In parts with just the sky use a big block.
• In detailed parts use smaller blocks.
 Something to think about…
Muren Board

 Silicon Recognition, Inc.

 2 ZISC078 Chips (156 neurons in
parallel)
 1 Mb of memory

 Up to 1 million recognitions/second
ZISC

 ZISC, the cornerstone of the Muren board,

completely overcomes the three major
limitations of software-based pattern
recognition applications:
 ZISC fully handles non-linearity and
fuzziness
 ZISC is a fully massively parallel processing
system. Recognition speed is consistent,
regardless of the number of stored patterns
 ZISC does not use any algorithms for
programming the solution, ZISC is taught
RBF Space mapping
Example

Source picture Result (after processing)

(circuit board) (shows good and bad points)
Process

 Learning
 Classifying

 Training

 Testing with new data

NASA Data
Further investigation

 Can we write directly to the board?

 Ability to preprocess data?
Questions?