E0 270 · Machine Learning

E0 270January–April 2018

Machine Learning

Department of Computer Science and Automation · Indian Institute of Science

Instructors

Prof. Chiranjib Bhattacharyya (chiru [at] iisc.ac.in)

Prof. Ambedkar Dukkipati (ad [at] iisc.ac.in)

Teaching assistants

Abhinav Anand

Annervaz K M

Shubham Gupta

Sweta Sharma

Lectures

Tuesday and Thursday, 11:00 AM – 12:30 PM, CSA 117 (first lecture: January 2, 2018)

Announcements

Course description

With the increasing amounts of data being generated in diverse fields such as astronomical sciences, health and life sciences, financial and economic modeling, climate modeling, market analysis, and even defense, there is an increasing need for computational methods that can automatically analyze and learn predictive models from such data.

Machine learning, the study of computer systems and algorithms that automatically improve performance by learning from data, provides such methods; indeed, machine learning techniques are already being used with success in a variety of domains, for example in computer vision to develop face recognition systems, in information retrieval to improve search results, in computational biology to discover new genes, and in drug discovery to prioritize chemical structures for screening.

This course aims to provide a sound introduction to both the theory and practice of machine learning, with the goal of giving students a strong foundation in the subject, enabling them to apply machine learning techniques to real problems, and preparing them for advanced coursework and research in machine learning and related fields.

Prerequisites

E0 232: Probability and Statistics (or an equivalent course elsewhere), with a grade of B or higher. Some background in linear algebra and optimization will also be helpful.

Evaluation (tentative)

Lectures

#DateTopicRecommended reading
1January 2Organisational meeting
2January 4Introduction to machine learning; classification using Bayes ruleDuda and Hart: Chapter 2
3January 11Introduction to Bayes decision theoryDuda and Hart: Chapter 2
4January 16Bayes decision theory (contd.)Duda and Hart: Chapter 2
5January 18Learning as optimization; linear regressionBishop: Chapter 3
6January 23Probabilistic view of linear regression: ML and MAP estimates
7January 30Logistic regression: optimization and probabilistic approaches
8February 1Parameter estimation for logistic regression: gradient descent, Newton's method, stochastic gradient methods
9February 6Hyperplane-based classifiers, the perceptron and the perceptron convergence theorem
10February 9Introduction to support vector machines
11February 13Support vector machines
12February 20Kernel methods and kernel SVM
13February 22Feed-forward neural networks and backpropagation
14February 28Undirected graphical models, Markov random fieldsAn Introduction to Restricted Boltzmann Machines
15March 4Introduction to MCMC and Gibbs samplingAn Introduction to Restricted Boltzmann Machines
16March 6Restricted Boltzmann machinesAn Introduction to Restricted Boltzmann Machines; A Practical Guide to Training Restricted Boltzmann Machines
17March 8Convolutional and recurrent neural networks
18March 13Boltzmann machines and variational methodsAckley, Hinton and Sejnowski (1985), A Learning Algorithm for Boltzmann Machines; Boltzmann Machines
19March 15Variational methods (contd.)Variational methods (see Section 6.3.1)
20March 22EM algorithm, mixture models and k-meansBishop: Chapter 9
21March 27Bayesian networks; introduction to HMMsA Tutorial on HMMs
22April 2HMMs (contd.): the forward–backward algorithmA Tutorial on HMMs

Projects

As part of this course, you are required to work on a project. This will give you hands-on experience of working with data from various domains (text, images, videos, etc.) used in contemporary ML research, and also expose you to specialized topics of ML that are more advanced or recent than what will be covered in the lectures. A list of project ideas will be shared with the class; you can also come up with your own project ideas.

Project policy

Assignment policy

References

Recommended textbooks

Additional textbooks

Optimization textbooks

Academic honesty

As students of IISc, we expect you to adhere to the highest standards of academic honesty and integrity.

Elements of the course are designed to support your learning of the subject. Copying will not help you (in the exams or in the real world), so don't do it. If you have difficulties learning some of the topics or lack some background, try to form study groups where you can bounce off ideas with one another and try to teach each other what you understand. You're also welcome to talk to any of us and we'll be glad to help you.

If any exam/assignment is found to be copied, it will automatically result in a zero grade for that assignment and a warning note to your advisor. Any repeat instance will automatically lead to a failing grade in the course.