Theory of Computation

ABOUT THE COURSE

This is an introductory course on Theory of Computation intended for undergraduate students
in computer science. In this course we will introduce various models of computation and
study their power and limitations. We will also explore the properties of the corresponding
language classes defined by these models and the relations between them. We will assume the
student is comfortable in analytical reasoning and has preferably done a course on Data
Structures and Algorithms.
INTENDED AUDIENCE
Computer Science undergraduate students.
PRE-REQUISITES
It is recommended that the candidate has done a course in Data Structures and Algorithms.
INDUSTRY SUPPORT LIST OF COMPANIES/INDUSTRY THAT WILL
RECOGNIZE/VALUE THIS ONLINE COURSE
content will be updated soon.

COURSE INSTRUCTOR
Dr. Raghunath Tewari is an Assistant Professor in the department of Computer Science and
Engineering at the Indian Institute of Technology, Kanpur. His primary research interest is in
the area of computational complexity theory. Dr. Tewari did his B.Sc. from Chennai
Mathematical Institute in 2005 and Ph.D. from University of Nebraska-Lincoln in 2011.

COURSE LAYOUT
Week 1: Finite Automata deterministic and nondeterministic, regular operations
Week 2: Regular Expression, Equivalence of DFA, NFA and REs, closure properties
Week 3: Non regular languages and pumping lemma, DFA Minimization,
Week 4: CFGs, Chomsky Normal Form
Week 5: Non CFLs and pumping lemma for CFLs, PDAs, Equivalence of PDA and CFG
Week 6: Properties of CFLs, DCFLs, Turing Machines and its variants
Week 7: Configuration graph, closure properties of decidable languages, decidability
properties of regular languages and CFLs
Week 8: Undecidability, reductions, Rice's Theorem, introduction to complexity theory

SUGGESTED READING
Introduction to the Theory of Computation by Michael Sipser.
MORE DETAILS ABOUT THE COURSE
Course url: https://onlinecourses.nptel.ac.in/noc16_cs14
Course duration : 8 Weeks
Start date and end date of course: 18 July 2016 - 9 September 2016
Dates of exams : 18 Sep 2016 and 25 Sep 2016
Time of exam : 2pm - 5pm
Final List of exam cities will be available in exam registration form.
Exam registration url - Will be announced shortly
Exam Fee: The online registration form has to be filled and the certification exam fee of
approximately Rs 1000(non-Programming) needs to be paid.

CERTIFICATE
E-Certificate will be given to those who register and write the exam. Certificate will have
your name, photograph and the score in the final exam. It will have the logos of NPTEL and
IIT Kanpur.
It will be e-verifiable at nptel.ac.in/noc.

An Introduction to Information Theory

ABOUT THE COURSE

Information Theory answers two fundamental questions: what is the maximum data rate at
which we can transmit over a communication link, and what is the fundamental limit of data
compression. In this course we will explore answers to these two questions. We will study
some practice source compression algorithms. We will also study how to compute channel
capacity of simple channels.
INTENDED AUDIENCE
3rd/4th year UG students in EC stream, 1st year PG students in communications and signal
processing specialization
PRE-REQUISITES
Basic knowledge of probability theory and digital communications
INDUSTRY SUPPORT LIST OF COMPANIES/INDUSTRY THAT WILL
RECOGNIZE/VALUE THIS ONLINE COURSE
Communication companies, defense laboratories
Adrish received his Bachelors degree from Indian Institute of Technology, Kharagpur and Masters
and Ph.D. degree from University of Notre Dame, Indiana. He is currently an Associate Professor in
the Department of Electrical Engineering at Indian Institute of Technology, Kanpur. He has been a
visiting faculty to National Yunlin University of Science and Technology, Taiwan and Chung-Ang
University, Seoul, South Korea. Under Erasmus- Mundus program he was a visiting faculty in
Politecnico di Torino, Italy. He is a recipient of Microsoft Research India young faculty award, and
Institute of Engineers India young engineer award. His research interests are in the physical layer
aspects of wireless communications, particularly error control coding, cognitive radio and green
communications.

COURSE LAYOUT
Week 1:
Introduction: Entropy, Relative Entropy, Mutual Information;
Information Inequalities;
Week 2:
Block to variable length coding-I: Krafts inequality;
Block to variable length coding-II: Huffman coding;
Variable to block length coding: Tunstall coding.

Week 3:
Block to block length coding: Typical sequences;
Variable to variable length coding-I: Arithmetic codes;
Variable to variable length coding-II: Lempel-Ziv codes.
Week 4:
Asymptotic Equipartition Property;
Coding for sources with memory.
Week 5:
Noisy channel coding theorem;
Converse of noisy channel coding theorem;
Channel capacity of discrete memoryless channels.
Week 6:
Differential entropy ;
Gaussian Channel;
Parallel Gaussian Channel.
Week 7:
Rate Distortion Theory;
Blahut-Arimoto Algorithm for computation of channel capacity and rate- distortion function.
Week 8:
An introduction to Network Information Theory-I;
An introduction to Network Information Theory-II.
SUGGESTED READING
1. James L. Massey, Lecture notes on ``Applied Digital Information Theory I''.
2. Thomas M. Cover, Joy A. Thomas, ``Elements of Information Theory'', 2nd Edition,
John Wiley & Sons, 2006.
3. Robert G. Gallager, ``Information Theory and Reliable Communications'', John Wiley
& Sons, 1968.
4. Raymond W. Yeung, ``Information Theory and Network Coding'', Springer, 2008.
5. David J. C. MacKay, ``Information Theory, Inference, and Learning Algorithms'',
Cambridge University Press.
6. Robert Ash, ``Information Theory'', Dover Publications, 1965.
7. Imre Csiszar and Jonos Korner, ``Information Theory'', Second edition, Cambridge
University Press, 2011.

MORE DETAILS ABOUT THE COURSE

Course url: https://onlinecourses.nptel.ac.in/noc16_ec18
Course duration: 8 weeks
Start date and end date of course: 18 July 2016 - 9 September 2016
Dates of exams:18 Sep 2016 and 25 Sep 2016
Time of exam : 2pm - 5pm
Final List of exam cities will be available in exam registration form.
Exam registration url - Will be announced shortly
Exam Fee: The online registration form has to be filled and the certification exam fee of
approximately Rs 1000(non-Programming) needs to be paid.

CERTIFICATE
E-Certificate will be given to those who register and write the exam. Certificate will have
your name, photograph and the score in the final exam. It will have the logos of NPTEL
and IIT Kanpur.
It will be e-verifiable at nptel.ac.in/noc.
Theory of computation
https://onlinecourses.nptel.ac.in/noc16_cs14/course#registration_confirmation
Information Theory
https://onlinecourses.nptel.ac.in/noc16_ec18/unit?unit=8&lesson=10