E0 270: Machine Learning, January Term 2020

E0 270: Machine Learning
  (Jan-April 2020)

Department of Computer Science & Automation
Indian Institute of Science



[Course Description]  [Projects]  [Academic Honesty]  [Syllabus] 

Course Information


Class Meetings

Lectures: Tuesday, Thursday 9:30am-11:00am, CSA 117
First lecture: Thu, Jan 2

Tutorial/discussion sessions will be scheduled on an on-going basis.

Instructors

Prof. Ambedkar Dukkipati (ambedkar@iisc)
Prof. Chiranjib Bhattacharyya (chiru@iisc)

Piazza link : https://piazza.com/iisc.ernet.in/spring2020/e0270
(Please use your IISc email address for registration)

TAs

Shubham Gupta [shubhamg@iisc.ac.in]
Tony Gracious [tonygracious@iisc.ac.in]
Shaarad A R [rangaa@iisc.ac.in]

TA Hours

Please take appointment over email from the TA

Announcements

Course Description

With the increasing amounts of data being generated in diverse fields such as astronomical sciences, health and life sciences, financial and economic modeling, climate modeling, market analysis, and even defense, there is an increasing need for computational methods that can automatically analyze and learn predictive models from such data. Machine learning, the study of computer systems and algorithms that automatically improve performance by learning from data, provides such methods; indeed, machine learning techniques are already being used with success in a variety of domains, for example in computer vision to develop face recognition systems, in information retrieval to improve search results, in computational biology to discover new genes, and in drug discovery to prioritize chemical structures for screening. This course aims to provide a sound introduction to both the theory and practice of machine learning, with the goal of giving students a strong foundation in the subject, enabling them to apply machine learning techniques to real problems, and preparing them for advanced coursework/research in machine learning and related fields.

Course Evaluation (Tentative)

1 Project - 20 marks
2 Mid Term Exam - 15 marks each
1 Final Exam - 50 marks

References

Recommended textbooks: Additional textbooks: (Optimization texbooks)

Preferred background

E0 232: Probability and Statistics (or equivalent course elsewhere) and earned a grade of B or higher. In addition, some background in linear algebra and optimization will be helpful.

Projects

As part of the this course, you are required to work on a project. This will give you hands-on experience of working with data from various domains - text, images, videos etc that are used in the contemporary ML research, and also expose you to some specialized topics of ML that are more advanced or recent than what will be covered in the lectures.

A list of project ideas will be mailed on the mailing list. You can also come up with your own projects ideas.

Project Policy

Course Project Presentation

Will be updated later

Academic Honesty

As students of IISc, we expect you to adhere to the highest standards of academic honesty and integrity.

Elements of the course are designed to support your learning of the subject. Copying will not help you (in the exams or in the real world), so don't do it. If you have difficulties learning some of the topics or lack some background, try to form study groups where you can bounce off ideas with one another and try to teach each other what you understand. You're also welcome to talk to any of us and we'll be glad to help you.

If any exam/report is found to be copied, it will automatically result in a zero grade for that exam/project and a warning note to your advisor. Any repeat instance will automatically lead to a failing grade in the course.


Syllabus



  Topic Recommended Reading
1 Introduction to Machine Learning, Classification using Bayes rule Duda and Hart: Chapter 2
2 Introduction to Bayes Decision Theory Duda and Hart: Chapter 2
3 Bayes Decision Theory (Contd...) Duda and Hart: Chapter 2
4 Learning as Optimization, Linear Regression Bishop: Chapter 3
5 Probabilistic view of Linear Regression: ML and MAP estimates NA
6 Logistic Regression: Optimization and Probabilistic approaches NA
8 Parameter estimation for Logistic Regression: Gradient Descent, Stochastic Gradient methods NA
7 Hyperplane based classifiers, Perceptron, and Perceptron Convergence Theorem NA
8 Introduction to Support Vector Machines NA
9 Support Vector Machines NA
10 Kernel Methods and Kernel SVM NA
11 Feedforward Neural Network and Backpropagation Algorithm NA
12 Undirected Graphical Models, Markov Random Fields An Introduction to Restricted Boltzmann Machines
13 Introduction to MCMC and Gibbs Sampling Chapter 11, Bishop Sections: 11.1.5, 11.2.1
Markov Chain Monte Carlo (MCMC)
14 Restricted Boltzmann Machine An Introduction to Restricted Boltzmann Machines
A Practical Guide to Training Restricted Boltzmann Machines
15 Boltzmann Machine and Variational methods https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog0901_7
Boltzmann Machines
16 Variational methods (contd...) Variational methods (See Section 6.3.1)
17 EM algorithm, Mixture models and K-means Bishop: Chapter 9
18 Bayesian Networks, Introduction to HMMs Tutorial on HMM
19 HMMs (contd...) - Forward Backward algorithm Tutorial on HMM
20 Introduction to Undirected Graphical Models and Markov Random Fields Chapter 8, Bishop
21 Variational Bayes and Variational autoencoders. Tutorial on Variational Autoencoders
22 Convolutional and Recurrent Neural Networks Tutorial on CNN and RNN
Understanding CNN for NLP
Understanding LSTM Networks