Module 4- 15CS81

Data and Analytics

for IoT
Rijo Jackson Tom, 9500 191 494, rijo.j@cmrit.ac.in
 05/03/2020
Don’t need to type
attendance in chat.
I will share a google
sheet. ……

Rijo Jackson Tom

2
 05/03/2020
NLP
Image Processing
Pattern recognition
Automation
Artificial

Rijo Jackson Tom

Intelligence

3
 05/03/2020
Artificial Intelligence
• “Thestudy of the modelling of human mental
functions by computer programs.” 
•2 words Artificial and intelligence
• created by humans is artificial
• Intelligence
means ability to understand,
reason, plan etc.

Rijo Jackson Tom

4
 Applications of AI

Rijo Jackson Tom 05/03/2020

5
 Timeline of Developments

Rijo Jackson Tom 05/03/2020

6
 05/03/2020
Real Life examples of AI before
2000
• Deep Blue was a
chess-playing computer developed
by IBM in 1996.
• It is known for being the first
computer chess-playing system to
win both a chess game and a
chess match against a reigning

Rijo Jackson Tom

world champion under regular time
controls.

7
 AI, defined it as “

05/03/2020
the science and engineering of making intelligent
machines
.” John McCarthy

The subsets
confused…..
A computer system able to
perform tasks that normally
require human intelligence,
such as visual perception,
speech recognition, decision-

Rijo Jackson Tom

making, and translation
between languages.

8
 05/03/2020
Differences
• AI can be a pile of if-then statements, or a complex statistical model
mapping raw sensory data to symbolic categories.
• The if-then statements are simply rules explicitly programmed by a
human hand.
• statements are sometimes called 
rules engines, expert systems, knowledge graphs or symbolic AI.
Collectively, these are known as Good, Old-Fashioned AI (GOFAI).
• But many don’t consider this as complete AI as it is programmed what
to be done.

Rijo Jackson Tom

• Machine Learning: Programs That Alter Themselves

9
 Rijo Jackson Tom 05/03/2020
10
 05/03/2020
Machine Learning: Programs
That Alter Themselves
• Machine learning is a subset of AI.
• That is, all machine learning counts as AI, but not all AI counts as
machine learning. 
• symbolic logic – rules engines, expert systems and knowledge graphs
– could all be described as AI, and none of them are machine learning.

• One aspect that separates machine learning from

the knowledge graphs and expert systems is its ability to modify
itself when exposed to more data; i.e. machine learning
is dynamic and does not require human intervention to make certain

Rijo Jackson Tom

changes. 

11
 05/03/2020
• “Machine learning is the science
of getting computers to act
without being explicitly
programmed.” 
• It’s a subset of AI which uses
statistical methods to enable
machines to improve with
experience.

Machine
Learning

Rijo Jackson Tom

12
 05/03/2020
Deep Learning: More Accuracy,
More Math & More Compute
• Deep artificial neural networks are a set of algorithms
that have set new records in accuracy for many important problems,
such as 
• image recognition,
• sound recognition,
• recommender systems, 
• natural language processing etc.

Rijo Jackson Tom

• Deep is a technical term. It refers to the number of layers in
a neural network. A shallow network has one so-called hidden
layer, and a deep network has more than one. 
13
 05/03/2020
Real Life examples of AI present
• AlphaGo is a computer program that plays the board
game Go
• developed by Alphabet Inc.'s Google DeepMind in
London
• October 2015, AlphaGo became the first computer Go
program to beat a human professional Go player
without handicaps on a full-sized 19×19 board.
• AlphaGo and its successors use a 
Monte Carlo tree search algorithm to find its moves
based on knowledge previously "learned" by 
machine learning, specifically by an 
artificial neural network (a deep learning method) by

Rijo Jackson Tom

extensive training, both from human and computer
play

14
 05/03/2020
Why this ML hype now?
A computer program
E- less data is said to learn from
T- more data experience E with
Hardware respect to some
power class of tasks T and
performance
measure P if its
performance at tasks
in T, as measured by
P, improves with
experience E. –
Tom Mitchell

Rijo Jackson Tom

15
 Internet of Things

Rijo Jackson Tom 05/03/2020

16
 Edge/Fog
Computing

Rijo Jackson Tom 05/03/2020

17
 IoT in Smart Grids

Rijo Jackson Tom 05/03/2020

18
 05/03/2020
Machine learns like a Child

Rijo Jackson Tom

19
Machine
Learning
 05/03/2020
Supervised Learning
Supervised Learning is the most popular learning
mechanism used today. more than 90% of the
Machine Learning algorithm used today are
Supervised.

Rijo Jackson Tom

21
 Unsupervised Learning

Rijo Jackson Tom 05/03/2020

22
 Reinforcement Learning

Rijo Jackson Tom 05/03/2020

23
 ML
Steps Involved

Rijo Jackson Tom 05/03/2020

24
 05/03/2020
Dataset split Thumb rule for ML
Percentage of Data
Training Data Validate Data Test data

[PERCE
NTAGE]
[PERCE
NTAGE] [PERC

Rijo Jackson Tom

ENTA
GE]

25
Machine Learning
Implementation
Tools for implementing machine learning
 05/03/2020
Ways to implement ML
• WEKA machine learning workbench (open source)

•R platform

• Scikit Learn package in Python

• Rapidminer (GUI based)

• MATLAB (statistics and machine learning toolbox)

Rijo Jackson Tom

• https://colab.research.google.com/notebooks/intro.ipynb

27
 05/03/2020
Python for Machine Learning
• Popular
general purpose programming
language
• From
basic level programming to
production level
• Special packages
 Numerical and statistics – numpy

Rijo Jackson Tom

 Plotting- matplotlib
 Relational database- pandas
 Machine learning – scikit-learn

28
 05/03/2020
Jupyter Notebooks
• Anaconda python installation package
• Jupyter notebooks
 Cell based execution

• Spyder-
Scientific Python Development
environment
• Anaconda
has other data visualization and

Rijo Jackson Tom

programming packages too.

29
 05/03/2020
Rijo Jackson Tom
https://scikit-learn.org/stable/ 30
 Rijo Jackson Tom 05/03/2020
31
Thank you!
Until next time
Regression
Supervised Learning – Simple Linear Regression, Polynomial
Regression
 05/03/2020
Supervised machine learning
• Setof labeled examples to learn from: training
dataset
• Develop model from training data
• Use model to predictions about the new data
• Training data
 Set of predictor values – Independent variables
 Numerical output value – Dependent variable

Rijo Jackson Tom

• Model is a function from predictors to output
 Use model to predict output value for new predictor values

34
 05/03/2020
Simple Linear Regression
• We will focus on:
 One numeric predictor value,
call it x
 One numeric output value, call
it y

• Data items are points in

two-dimensional space
• Plot x and y

Rijo Jackson Tom

35
 05/03/2020
Simple Linear Regression
•A function f(x)=y is a line now
• the function y= a*x + b

y= 0.8 x + 2.6

Rijo Jackson Tom

36
 05/03/2020
Simple Linear Regression
• Real Examples

Rijo Jackson Tom

37
 05/03/2020
Correlation and Causation
• Correlation- Values track each other
 Height and shoe size
 Grades and entrance exam score
• Causation-one value directly
influences the others
 Education level – starting salary

Rijo Jackson Tom

 Temperature – cold drink sales

38
 05/03/2020
Regression and Correlation
• The
better the function fits the points,
the more correlated x and y are.
• Linear functions only
• Correlation- values track each other
 Positive- when one goes up other also goes
up

Rijo Jackson Tom

 Negative- when one goes up the other goes
down
39
 05/03/2020
Calculating Simple linear
Regression
• Method of least squares
• Given a point and a line, the
error for the point is its
vertical distance d from the
line, and the squared error
is d2
• Given a set of points and a
line, the sum of squared
error (SSE) is the sum of the
squared errors for all the

Rijo Jackson Tom

points
• Goal: Given a set of points,
find the line that minimizes
the SSE

40
 05/03/2020
Calculating Simple linear
Regression

Rijo Jackson Tom

41
 05/03/2020
Measuring correlation
• Pearson’s Product Moment Correlation (PPMC)
 “Pearson’s coefficient”, “ correlation coefficient”
 Value of r between 1 and -1
 1 for maximum positive correlation
 0 for no correlation
 -1 for maximum negative correlation

• Coefficient of Determination
 R2, r2, “ R squared”

Rijo Jackson Tom

 Measures the fit of any line/ curve to set of points
 Usually between 0 and 1
 For simple linear regression R2 = Pearson2

42
 05/03/2020
Regression through spreadsheets
• City temperatures (using Cities.csv)
temperature (y) versus latitude (x)

temperature (y) versus longitude (x)

latitude (y) versus longitude (x)

longitude (y) versus latitude (x)

Rijo Jackson Tom

43
 Drawbacks

Rijo Jackson Tom 05/03/2020

44
 Polynomial Regression

Rijo Jackson Tom 05/03/2020

45
 05/03/2020
Drawbacks of polynomial
over fitting and under fitting

Rijo Jackson Tom

46
 05/03/2020
Regression summary
• Supervised learning
• Training dataset
• Modelis a function form inputs to
outputs
• Find the best fit

Rijo Jackson Tom

• Drawbacks chances of over fitting.

47
Classification
Supervised Learning - K Nearest Neighbours, Decision Tree
 05/03/2020
Classification
• Setof feature values numerical or
categorical
• Categorical – labels
• Model formed from features to labels
• Example

Rijo Jackson Tom

 Features: age, gender, income, profession
 Label: buyer, non-buyer

49
 05/03/2020
Other applications
• Medical Diagnosis
 Feature values: age, gender, history, symptom1-
severity, symptom2-severity, test-result1, test-
result2
 Label: disease
• Email spam detection
 Feature values: sender-domain, length, images, keyword1,
keyword2, …, keywordn
 Label: spam or not-spam

Rijo Jackson Tom

• Credit card fraud detection
 Feature values: user, location, item, price
 Label: fraud or okay

50
 05/03/2020
K-Nearest Neighbour (KNN)

Rijo Jackson Tom

51
 05/03/2020
K-Nearest Neighbour (KNN)

Rijo Jackson Tom

52
 05/03/2020
K-Nearest Neighbour (KNN)

Rijo Jackson Tom

53
 KNN examples

Rijo Jackson Tom 05/03/2020

54
 KNN
examples

Rijo Jackson Tom 05/03/2020

55
 Summary

Rijo Jackson Tom 05/03/2020

56
 Decision Trees

Rijo Jackson Tom 05/03/2020

57
 Rijo Jackson Tom 05/03/2020
58
Clustering
Unsupervised learning – K means Clustering
 05/03/2020
Unsupervised Machine learning
• Unlabeled data
• Look for patterns or structures

Rijo Jackson Tom

60
 Supervised Learning

Rijo Jackson Tom 05/03/2020

61
 Rijo Jackson Tom 05/03/2020
62
Deep Learning
 05/03/2020
Deep Learning
• “Deep Learning is a subfield of machine learning concerned with
algorithms inspired by the structure and function of the brain called
artificial neural networks”

Rijo Jackson Tom

64
 Rijo Jackson Tom 05/03/2020
65
 Rijo Jackson Tom 05/03/2020
66