Convolutional Neural Network (CNN) is used for processing images, audio, and video.

For
simple CNNs, data is usually a 2-D file thus input being represented as a square.
Each neuron in a convolution layer is responsible for a small cluster of neurons in the
preceding layer. The bounding box that determines the cluster of neurons is called a filter,
also known as a kernel. Conceptually, you can think of a filter moving across the image and
performing a mathematical operation on individual regions of the image. It then sends the
result to the corresponding neuron in the convolution layer.
Pooling reduces the number of neurons in the previous convolution layer while still retaining
the most important information. There are different types of pooling that can be performed.
For example, taking the average of each input neuron, the sum of each neuron, or the
maximum value.
When we put all these techniques together, we get an architecture for a deep neural
network quite different from our fully connected neural network. First, we take an input
image, which is a two-dimensional matrix, typically with three color channels. Next, we use a
convolution layer with multiple filters to create a two-dimensional feature matrix as output
for each filter. Then, we pool the results to produce a down sample feature matrix for each
filter in the convolution layer. Next, we typically repeat the convolution and pooling steps
multiple times using previous features as input. Then, we add a few fully connected hidden
layers to help classify the image, and finally, we produce our classification prediction in the
output layer.
All of the neural networks we've seen so far have been feedforward neural networks. They
are called this because data flows only from the input x through one or more hidden
neurons, h, to the output y. However, we also have several types of neural network
architectures that contain feedback loops. Unlike feedforward neural networks, the
recurrent neural network, or RNN, can operate effectively on sequences of data with
variable input length.
We're just going to focus on a single path through the network from the input x, through the
hidden neuron h, to the output neuron y. Now let's unfold this path through the neural
network over time. If we imagine this RNN moving through time, this first path represents a
network in time step one. The hidden node h1 uses the input x1 to reduce output y1. This is
exactly what we've already seen with basic feedforward neural networks. Now let's add a
second time step. The hidden node at the current time step, h2, uses both the new input x2,
and its state from the previous time step, h1, as input to make its new prediction, y2. This
means that a recurrent neural network uses knowledge of its previous state as an input for
its current prediction, and we can repeat this process for an arbitrary number of steps
allowing the network to propagate information via its hidden state through time.
There are few variants on the basic RNN architecture that help solve a common problem
with training RNNs known as vanishing and exploding gradients. Two of these variants are
Gated RNNs, and Long Short-Term Memory RNNs, also known as LSTMs. Both of these
variants use a form of memory to help make predictions in sequences over time. The main
difference between a Gated RNN and an LSTM is that the Gated RNN has two gates to
control its memory: an Update gate and a Reset gate, while an LSTM has three gates: an
Input gate, an Output gate, and a Forget gate.

The next deep neural network architecture that we need to discuss is a Generative
Adversarial Network, or GAN for short. The GAN is a combination of two deep learning
neural networks: a Generator Network, and a Discriminator Network. The Generator
Network produces synthetic data, and the Discriminator Network tries to detect if the data
that it's seeing is real or synthetic. These two networks are adversaries in the sense that
they're both competing to beat one another. The Generator is trying to produce synthetic
data indistinguishable from real data, and the Discriminator is trying to become
progressively better at detecting fake data.
Now let's discuss reinforcement learning: our final technique for creating deep neural
networks that can solve a variety of problems. Reinforcement learning involves an agent
interacting with an environment. The agent is trying to achieve a goal of some kind within
the environment. The environment has state, which the agent can observe. The agent has
actions that it can take, which modify the state of the environment, and the agent receives
reward signals when it achieves a goal of some kind. The objective of the agent is to learn
how to interact with its environment in such a way that allows it to achieve its goals.

For example, an agent might be a car trying to get its passengers to their destination. The
environment would be the world the car is driving in. This would include the road, other
cars, pedestrians, and any obstacles on the road. The car can observe the state of its
environment; for example, it's position, speed, and direction, the orientation of the road it's
driving on, and the location of any obstacles in its path. The car has actions that it can
perform to modify the state of the world. For example, the car can accelerate, decelerate,
turn left, or turn right. This allows it to change its position relative to the objects in the
world. The car would receive reward signals when it achieves a goal of some kind; for
example, we could reward the car when it arrives safely at its destination, for each mile it
stays on the road, and for each minute it drives at a safe speed. We could also penalize the
car when it drives off the road, travels at unsafe speeds, or climbs with an obstacle. The
objective for the car is to learn how to drive in this world in such a way that it arrives at its
destination.

Deep reinforcement learning is the application of reinforcement learning to train deep

neural networks. Like our previous deep neural networks, we have an input layer, an output
layer, and multiple hidden layers. However, our input is the state of the environment; for
example, position, speed, and direction; our output is a series of a possible actions; for
example, speed up, slow down, turn left, or turn right. In addition, we're feeding our
rewards signal into the network so that we can learn to associate what actions produce
positive results given a specific state of the environment. This deep neural network attempts
to predict the expected future reward for each action, given the current state of the
environment. It then chooses whichever action's predicted to have the highest potential
future reward, and performs that action
Some examples of Deep Reinforcement Learning Applications are games, including board
games like chess and Go, card games like poker, and 8-bit video games. Autonomous
vehicles, like self-driving cars, and autonomous drones. Robotics, including teaching robots
how to walk, and teaching robots how to perform manual tasks. Management tasks,
including inventory management, resource allocation, and logistics; and financial tasks,
including investment decisions, portfolio design, and asset pricing.

Classification is where we attempt to make a decision or a prediction involving two or more

categories or outcomes. For example, deciding whether to accept or reject a loan based on
data from a customer's financial history. Second, we have regression. Regression is where we
attempt to predict a numeric outcome based on one or more input variables. For example,
trying to predict the sale price of a house based on the features of the house compared to
the sale price of similar houses. Third, we have clustering. Clustering is where we attempt to
group similar objects together based on similarities in their data. For example, grouping
customers into marketing segments based on their income, age, gender, number of children,
etc. Finally, we have anomaly detection. Anomaly detection is where we find observations in
the data that are different from the normal data. For example, detecting an unusual spike in
the number of negative comments about a product that we've just released. While all four
of these tasks can be performed on tabular data using deep neural networks, oftentimes
deep learning is overkill for these types of tasks. This is typically the case when the tabular
dataset is small, or the function we are attempting to model is relatively simple. These types
of problems are generally solved more effectively using traditional machine learning tools,
for example, decision tree classifiers, support vector machines, k-means clustering, or
shallow, rather than deep neural networks. However, there are cases where the tabular
datasets are large enough, and the function that we are attempting to model is complex
enough that a deep neural network makes sense. For example, just days before I recorded
this module, Stanford University announced that it had created a deep neural network to
predict whether patients admitted to a hospital will die within the next year or not. The
deep neural network was able to predict patient mortality with approximately 90% accuracy.
This training algorithm used over 13, 000 columns and over 220, 000 rows of tabular data as
input in order to reach this level of prediction accuracy.