E0 270: Machine Learning, January Term 2018

E0 270: Machine Learning (Jan-April 2018)

Department of Computer Science & Automation
Indian Institute of Science

[Announcements] [Course Description] [Assignments] [Academic Honesty] [Tutorials/Discussions] [Schedule]
We have sent an invitation mail for Canvas. Please follow the instructions given in that email to create an account. Make sure that you edit your profile and enter your name after creating the account.

Feel free to write to [shubhamg@iisc] if you have not received the invitation (check your spam folder first, if you are using a Focussed Inbox on Outlook then check the 'Others' tab in inbox) or if you have any other difficulties logging in.

We will make all subsequent announcements via Canvas, hence registration is mandatory for people who are crediting the course.

Course Information

Class Meetings

Lectures: Tue-Thu 11:00am-12:30pm, CSA 117
First lecture: Tue Jan 2

Tutorial/discussion sessions will be scheduled on an on-going basis.

Instructors

Prof. Chiranjib Bhattacharyya (chiru@iisc)
Prof. Ambedkar Dukkipati (ad@iisc)

TAs

Abhinav Anand [abhinava@iisc]
Annervaz K M [annervaz@iisc]
Shubham Gupta [shubhamg@iisc]
Sweta Sharma [swetasharma@iisc]

TA Hours

Please take appointment over email from the TA

Announcements

Course Description

With the increasing amounts of data being generated in diverse fields such as astronomical sciences, health and life sciences, financial and economic modeling, climate modeling, market analysis, and even defense, there is an increasing need for computational methods that can automatically analyze and learn predictive models from such data. Machine learning, the study of computer systems and algorithms that automatically improve performance by learning from data, provides such methods; indeed, machine learning techniques are already being used with success in a variety of domains, for example in computer vision to develop face recognition systems, in information retrieval to improve search results, in computational biology to discover new genes, and in drug discovery to prioritize chemical structures for screening. This course aims to provide a sound introduction to both the theory and practice of machine learning, with the goal of giving students a strong foundation in the subject, enabling them to apply machine learning techniques to real problems, and preparing them for advanced coursework/research in machine learning and related fields.

Course Evaluation (Tentative)

1st Mid Term - Feb 15
2nd Mid Term - March 20

References

Recommended textbooks:

C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
R.O. Duda, P.E. Hart and D.G. Stork. Pattern Classification. Wiley-Interscience, 2nd Edition, 2000.
Simon Haykin, Neural Network A Comprehensive Foundation. Pearson, 2004

Additional textbooks:

T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2nd Edition, 2009. [free download available]
T. Mitchell, Machine Learning. McGraw Hill, 1997.

(Optimization texbooks)

Preferred background

E0 232: Probability and Statistics (or equivalent course elsewhere) and earned a grade of B or higher. In addition, some background in linear algebra and optimization will be helpful.

Projects

As part of the this course, you are required to work on a project. This will give you hands-on experience of working with data from various domains - text, images, videos etc that are used in the contemporary ML research, and also expose you to some specialized topics of ML that are more advanced or recent than what will be covered in the lectures.

A list of project ideas will be mailed on the mailing list. You can also come up with your own projects ideas.

Project Policy

Some projects ideas are based on reading and survey of several papers, and others are based on implementation of one or two papers, and comparison of the methods proposed in them over datasets. We will point you to relevant papers and datasets. You can read/implement other relevant papers and datasets if you want. You can also innovate and come up with your own approaches for solving the problems.
The best grades will be reserved for the groups which come up with some new approaches, either theoretically or experimentally, which have not been used in the literature.
Each project will have an associated project mentor. He/she will help you to locate papers and datasets, and clear doubts.
In implementation-based projects, it may so happen that codes are already available online. It is perfectly fine to use them. Only, that should be cited in the reports.
You are required to provide a mid-term report around the third week of March, and the final report shortly before the final examination.
Plagiarism is strictly prohibited while writing the reports. Any plagiarism detected will result in F grade for the course.

Assignments

Assignment Policy

The following assignment policy will be strictly followed:

Assignments are for your learning. You can work in groups, but you must write your own code and solutions. See academic honesty policy below.
Late submission, late by up to 2 days (with advance email to TA), are allowed with penalty. After one late submission, no further late submissions will be allowed.*

* Except in the case of a documented medical/personal emergency, which must be supported by a medical certificate or signed letter submitted to the instructors and to the CSA office.

Academic Honesty

As students of IISc, we expect you to adhere to the highest standards of academic honesty and integrity.

Assignments in the course are designed to support your learning of the subject. Copying will not help you (in the exams or in the real world), so don't do it. If you have difficulties learning some of the topics or lack some background, try to form study groups where you can bounce off ideas with one another and try to teach each other what you understand. You're also welcome to talk to any of us and we'll be glad to help you.

If any assignment/exam is found to be copied, it will automatically result in a zero grade for that assignment/exam and a warning note to your advisor. Any repeat instance will automatically lead to a failing grade in the course.

Lectures

	Date	Topic	Lecture Notes	Additional Recommended Reading	Notes
1	Jan 2	Organisational Meeting	NA	NA
2	Jan 4	Introduction to Machine Learning, Classification using Bayes rule	NA	Duda and Hart: Chapter 2
3	Jan 11	Introduction to Bayes Decision Theory	NA	Duda and Hart: Chapter 2
4	Jan 16	Bayes Decision Theory (Contd...)	NA	Duda and Hart: Chapter 2
5	Jan 18	Learning as Optimization, Linear Regression	NA	Bishop: Chapter 3
6	Jan 23	Probabilistic view of Linear Regression: ML and MAP estimates	NA	NA
7	Jan 30	Logistic Regression: Optimization and Probabilistic approaches	NA	NA
8	Feb 1	Parameter estimation for Logistic Regression: Gradient Descent, Newton Method, Stochastic Gradient methods	NA	NA
9	Feb 6	Hyperplane based classifiers, Perceptron, and Perceptron Convergence Theorem	NA	NA
10	Feb 9	Introduction to Support Vector Machines	NA	NA
11	Feb 13	Support Vector Machines	NA	NA
12	Feb 20	Kernel Methods and Kernel SVM	NA	NA
13	Feb 22	Feedforward Neural Network and Backpropagation Algorithm	NA	NA
14	Feb 28	Undirected Graphical Models, Markov Random Fields	NA	An Introduction to Restricted Boltzmann Machines
15	March 4	Introduction to MCMC and Gibbs Sampling	NA	An Introduction to Restricted Boltzmann Machines
16	March 6	Restricted Boltzmann Machine	NA	An Introduction to Restricted Boltzmann Machines A Practical Guide to Training Restricted Boltzmann Machines
17	March 8	Convolutional Neural Networks and Recurrent Neural Networks	NA	NA
18	March 13	Boltzmann Machine and Variational methods	NA	https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog0901_7 Boltzmann Machines
19	March 15	Variational methods (contd...)	NA	Variational methods (See Section 6.3.1)
20	March 22	EM algorithm, Mixture models and K-means	NA	Bishop: Chapter 9
21	March 27	Bayesian Networks, Introduction to HMMs	NA	Tutorial on HMM
22	April 2	HMMs (contd...) - Forward Backward algorithm	NA	Tutorial on HMM