Kasegaon Education Society (KES)

Rajarambapu Institute of Technology (RIT)

DEPARTMENT OF INFORMATION TECHNOLOGY

FMTH0301/Rev.5.1
Course Plan
Semester: 7 - Semester Year: 2019
Course Title: Parallel Computing Course Code: IT4032
Total Contact Hours: 3 Duration of ESE: 2 Hours
ESE Marks: 50 ISE & MSE Marks: 50
Lesson Plan Author: Mr. Akhilesh Mokashi Last Modified Date: 17-12-
2017
Checked By: Mr. Akhilesh Mokashi Last Reviewed Date: 15-08-
2018

Course Outcomes (COs):

At the end of the course the student should be able to:
1. Summarize parallel programming technique and compare it with sequential programming
2. Write parallel programs using mpi, open mp, cuda etc.
3. Describe cuda architecture with their memory structure
4. Compare parallel matrix algorithm
5. Explain parallel sorting algorithms

Powered by www.ioncudos.com Page 1 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Course Articulation Matrix: Mapping of Course Outcomes (COs) with Program

Outcomes (POs)
Course Title: Parallel Computing Semester: 7 - Semester
Course Code: IT4032 Year: 2019

Course Outcomes (COs) a b c d e f g h i j k l P P

/ Program Outcomes S S
(POs)
O- O-
1 2
1. Summarize parallel 1 1
programming technique
and compare it with
sequential programming
2. Write parallel 1 1 3
programs using mpi,
open mp, cuda etc.
3. Describe cuda 1 3 1
architecture with their
memory structure
4. Compare parallel 1 1 1
matrix algorithm
5. Explain parallel sorting 1 1 1 3
algorithms

Course Content
Course Code: IT4032 Course Title: Parallel Computing
L-T-P: 3-0-0 Credits: 3 Contact Hrs: 3
ISE & MSE Marks: 50 ESE Marks: 50 Total Marks: 100
Teaching Hrs: 3 Exam Duration: 2
hrs

Content Hrs
Unit - 1
Chapter No. 1 - Introduction 6.00
Implicit Parallelism, Limitations of Memory, Dichotomy of Parallel Computing hrs
Platforms, Physical Organization of Parallel Platforms, Communication Costs in
Parallel Machines, Routing Mechanisms for Interconnection Networks, Impact of
Process-Processor Mapping and Mapping Techniques.
Chapter No. 2 - Message-Passing Paradigm 7.00
Principles of Message-Passing Programming Send and Receive Operations, MPI: hrs
Topologies and Embedding, Overlapping Communication with Computation.
Shared memory model- OpenMP: OpenMP Programming model, Concurrent tasks
in OpenMP, Constructs in OpenMP, Data Handling, Open MP Library functions,
Environment variable.
Chapter No. 3 - Introduction to GPU Computing and CUDA 6.00
Introduction to GPU Computing, CUDA Data Parallelism Model, CUDA Program hrs
Structure, Device Memories and Data Transfer, Kernel Functions and Threading.

Powered by www.ioncudos.com Page 2 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

CUDA Threads: CUDA Thread Organization, Using blockIdx and threadIdx,

Thread Assignment, Thread Scheduling and Latency Tolerance.
Chapter No. 4 - Analytical Modeling 6.00
Sources of Overhead, Performance Metrics, The Effect of Granularity on hrs
Performance, Scalability of Parallel System, Minimum Execution Time and
Minimum Cost-Optimal Execution Time
Chapter No. 5 - Dense Matrix Algorithms 5.00
Matrix-Vector Multiplication, Matrix-Matrix Multiplication, Solving a System of hrs
Linear Equations
Chapter No. 6 - Sorting 6.00
Issues, Sorting Networks, Bubble Sort and its Variants, Quicksort, Bucket and hrs
Sample Sort

Powered by www.ioncudos.com Page 3 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Text Books (List of books as mentioned in the approved syllabus)

1. Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar, Introduction to Parallel
Computing, Second Edition , Pearson Education
Chapterwise Plan
Course Code and Title: IT4032 / Parallel Computing
Chapter Number and Title: 1 - Introduction Planned Hours: 6.00
hrs

Learning Outcomes:-
At the end of the topic the student should be able to:
Topic Learning Outcomes COs BL
1 Summarize parallel programming technique and compare it with CO2 L1,L
Sequential Programming 2,L3

Lesson Schedule
Lecture No. - Portion Planned Delivery Date Actual Delivery Date
covered per hour
1. 03-
Intro 07-
ducti 2018
on to
parall
el
comp
uting
2. 05-
Impli 07-
cit 2018
Parall
elism,
Limit
ations
of
Mem
ory
3. 07-
Dich 07-
otom 2018
y of
Parall
el
Com
putin
g
Platfo
rms,

Powered by www.ioncudos.com Page 4 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Physi
cal
Orga
nizati
on of
Parall
el
Platfo
rms
4. 10-
Com 07-
muni 2018
catio
n
Costs
in
Parall
el
Mach
ines
5. 12-
Routi 07-
ng 2018
Mech
anism
s for
Interc
onnec
tion
Netw
orks
6. 14-
Impa 07-
ct of 2018
Proce
ssPro
cesso
r
Mapp
ing
and
Mapp
ing
Tech
nique
s.

Review Questions

Powered by www.ioncudos.com Page 5 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Sl.No. - Questions TLOs BL

1. NULL Identify applications of parallel computing. TLO1
2. Compute processor peak performance NULL TLO1 L3
3. Explain superscalar execution NULL TLO1
4. NULL Demonstrate impact of cache in processor performance TLO1 L2

Powered by www.ioncudos.com Page 6 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Course Code and Title: IT4032 / Parallel Computing

Chapter Number and Title: 2 - Message-Passing Paradigm Planned Hours: 7.00
hrs

Learning Outcomes:-
At the end of the topic the student should be able to:
Topic Learning Outcomes COs BL
1 Write MPI Programs CO2 L3
2 Write OpenMP Programs CO3 L2

Lesson Schedule
Lecture No. - Portion Planned Delivery Date Actual Delivery Date
covered per hour
7. 17-
Princi 07-
ples 2018
of
Mess
age
8. 19-
Send 07-
and 2018
Recei
ve
Oper
ations
9. 21-
MPI 07-
2018
10. 24-
Open 07-
MP 2018
Progr
ammi
ng
mode
l
11. 26-
Conc 07-
urrent 2018
tasks
in
Open
MP
12. 28-
Const 07-

Powered by www.ioncudos.com Page 7 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

ructs 2018
in
Open
MP
13. 31-
Open 07-
MP 2018
Libra
ry
functi
ons

Review Questions
Sl.No. - Questions TLOs BL
1. NULL Write MPI programs TLO1 L3
2. NULL Define concurrent tasks in Open MP TLO2 L2
3. Write Open MP programms NULL TLO1 L3

Powered by www.ioncudos.com Page 8 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Course Code and Title: IT4032 / Parallel Computing

Chapter Number and Title: 3 - Introduction to GPU Computing Planned Hours: 6.00
and CUDA hrs

Learning Outcomes:-
At the end of the topic the student should be able to:
Topic Learning Outcomes COs BL
1 Write CUDA programs CO3 L3

Lesson Schedule
Lecture No. - Portion Planned Delivery Date Actual Delivery Date
covered per hour
14. 02-
Intro 08-
ducti 2018
on to
GPU
Com
putin
g
15. 04-
CUD 08-
A 2018
Data
Parall
elism
Mode
l,
CUD
A
Progr
am
Struct
ure
16. 07-
Devic 08-
e 2018
Mem
ories
and
Data
Trans
fer,
Kern
el
Funct
ions

Powered by www.ioncudos.com Page 9 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

and
Threa
ding
17. 14-
CUD 08-
A 2018
Threa
ds
18. 16-
Threa 08-
d 2018
Assig
nmen
t
19. 18-
Threa 08-
d 2018
Sche
dulin
g

Review Questions
Sl.No. - Questions TLOs BL
1. NULL Differentiate GPU and CPU TLO1 L3
2. NULL Write CUDA Program TLO1 L3
3. NULL Write program for global memory transfer\n Write program for TLO1 L3
memory utilization
4. Write a program to demonstrate thread scheduling NULL TLO1 L3

Powered by www.ioncudos.com Page 10 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Course Code and Title: IT4032 / Parallel Computing

Chapter Number and Title: 4 - Analytical Modeling Planned Hours: 6.00
hrs

Learning Outcomes:-
At the end of the topic the student should be able to:
Topic Learning Outcomes COs BL
1 Define Cost Optimal system CO4 L3

Lesson Schedule
Lecture No. - Portion Planned Delivery Date Actual Delivery Date
covered per hour
20. 21-
Sourc 08-
es of 2018
Over
head
21. 23-
Perfo 08-
rman 2018
ce
Metri
cs
22. 25-
The 08-
Effec 2018
t of
Gran
ularit
y
23. 28-
Scala 08-
bility 2018
of
Parall
el
Syste
m
24. 30-
Mini 08-
mum 2018
Exec
ution
Time
25. 01-
Cost- 09-
Opti 2018

Powered by www.ioncudos.com Page 11 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

mal
Exec
ution
Time

Review Questions
Sl.No. - Questions TLOs BL
1. NULL What are the sources of overhead compare with sequential TLO1
algorithm
2. What is granularity? How do you build it. NULL TLO1 L3
3. Derive the equation for minimum execution time NULL TLO1 L3
4. When do you say the system is cost optimal, explain with example TLO1 L3
NULL

Powered by www.ioncudos.com Page 12 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Course Code and Title: IT4032 / Parallel Computing

Chapter Number and Title: 5 - Dense Matrix Algorithms Planned Hours: 5.00
hrs

Learning Outcomes:-
At the end of the topic the student should be able to:
Topic Learning Outcomes COs BL
1 Matrix-Matrix Multiplication CO4 L4
2 Solving LPP CO4 L3,L
4

Lesson Schedule
Lecture No. - Portion Planned Delivery Date Actual Delivery Date
covered per hour
26. 04-
Matri 09-
x- 2018
Vecto
r
Multi
plicat
ion
27. 06-
Matri 09-
x- 2018
Matri
x
Multi
plicat
ion
28. 08-
Matri 09-
x- 2018
Matri
x
Multi
plicat
ion1
29. 11-
Solvi 09-
ng a 2018
Syste
m of
Linea
r
Equat
ions

Powered by www.ioncudos.com Page 13 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

30. 15-
Solvi 09-
ng a 2018
Syste
m of
Linea
r
Equat
ions1

Review Questions
Sl.No. - Questions TLOs BL
1. Explain matrix - vector multiplication with example NULL TLO1 L4
2. Explain matrix - matrix multiplication with example NULL TLO1 L4
3. Perform 1-D and 2-D partitioning of the matrix – matrix multiplication TLO2 L4
NULL
4. Solve linear equations by LU decomposition NULL TLO2 L3

Powered by www.ioncudos.com Page 14 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

Course Code and Title: IT4032 / Parallel Computing

Chapter Number and Title: 6 - Sorting Planned Hours: 6.00
hrs

Learning Outcomes:-
At the end of the topic the student should be able to:
Topic Learning Outcomes COs BL
1 Issues of sorting network CO4 L2
2 Parallel sorting algorithms CO5 L3,L
4

Lesson Schedule
Lecture No. - Portion Planned Delivery Date Actual Delivery Date
covered per hour
31. 22-
Issue 09-
s 2018
32. 04-
Sorti 10-
ng 2018
Netw
orks
33. 06-
Bubb 10-
le 2018
Sort
34. 09-
Quic 10-
k sort 2018
35. 11-
Buck 10-
et 2018
sort
36. 13-
Samp 10-
le 2018
Sort

Review Questions
Sl.No. - Questions TLOs BL
1. What are the issues to perform sorting on parallel computers NULL TLO2 L3
2. - Formulate bitonic sequence\n - Explain comparators Explain mapping TLO1 L2
of bitonic network with example NULL
3. Demonstrate variants of the bubble sort NULL TLO1 L2
4. What is bucket sort, how to define bucket size and how to parallelize TLO2 L4
bucket sort NULL

Powered by www.ioncudos.com Page 15 of 16.

 Kasegaon Education Society (KES)
Rajarambapu Institute of Technology (RIT)
DEPARTMENT OF INFORMATION TECHNOLOGY

5. Demonstrate sample sort with example NULL TLO1 L3

Powered by www.ioncudos.com Page 16 of 16.