With the increasing amounts of data being generated in diverse fields such as astronomical sciences, health and life sciences, financial and economic modeling, climate modeling, market analysis, and even defense, there is an increasing need for computational methods that can automatically analyze and learn predictive models from such data. Machine learning, the study of computer systems and algorithms that automatically improve performance by learning from data, provides such methods; indeed, machine learning techniques are already being used with success in a variety of domains, for example in computer vision to develop face recognition systems, in information retrieval to improve search results, in computational biology to discover new genes, and in drug discovery to prioritize chemical structures for screening. This course aims to provide a sound introduction to both the theory and practice of machine learning, with the goal of giving students a strong foundation in the subject, enabling them to apply machine learning techniques to real problems, and preparing them for advanced coursework/research in machine learning and related fields.

E0 232: Probability and Statistics (or equivalent course elsewhere) and earned a grade of B or higher.
In addition, some background in linear algebra and optimization will be helpful.

As students of IISc, we expect you to adhere to the highest standards of academic honesty and integrity.

Elements of the course are designed to support your learning of the subject. Copying will not help you (in the exams or in the real world), so don't do it. If you have difficulties learning some of the topics or lack some background, try to form study groups where you can bounce off ideas with one another and try to teach each other what you understand. You're also welcome to talk to any of us and we'll be glad to help you.

If any exam/report is found to be copied, it will automatically result in a zero grade for that exam/project and a warning note to your advisor. Any repeat instance will automatically lead to a failing grade in the course.

Assignment 1: PCA and SVM on MNIST

1. Please find attached the zip containing your first assignment. SVM_Assignment1.zip
2. Assignment description, task instructions, and submission instruction have been specified in a pdf file inside the zip file. You are required to send only 1 no. of zip file corresponding to your submission in the format: Asst1_FirstName_Last5digitSRNo.zip by email to e0.270.iisc.2023@gmail.com with the Subject: Asst1_FirstName_Last5digitSRNo
3. Due date is one week from today, i.e. March 29, 2023 (11:59 PM). If you are not crediting the course, feel free not to submit your assignment.
4. Setting up the environment, running into technical problems, and figuring out their solutions is part of the learning process. Use Google, StackOverflow, and watch youtube videos. Unfortunately we will not be able to assist you.
5. The report need not be typeset in LATEX. However, ensure that everything is in a single pdf document and pages are in the correct order.
6. If you feel any portion of the problem is under-specified, you can safely assume that it has been done intentionally. You are free to make assumptions, but please specify the assumptions you make in the report. No further clarifications on the problems will be provided. However, feel free to contact the TAs in case you believe that there exists an error in the problem specification.

Assignment 2: KMeans

1. Please find attached the zip containing your second assignment. KMeans_Assignment2.zip
2. Assignment description, task instructions, and submission instructions have been specified in the pdf file. You are required to send only 1 no. of zip file corresponding to your submission in the format: Asst2_FirstName_Last5digitSRNo.zip by email to e0.270.iisc.2023@gmail.com with the Subject: Asst2_FirstName_Last5digitSRNo
3. Due date is one week from today, i.e. April 12, 2023 (11:59 PM). If you are not crediting the course, feel free not to submit your assignment.
4. Setting up the environment, running into technical problems, and figuring out their solutions is part of the learning process. Use Google, StackOverflow, and watch youtube videos. Unfortunately we will not be able to assist you.
5. The report need not be typeset in LATEX. However, ensure that everything is in a single pdf document and pages are in the correct order.
6. If you feel any portion of the problem is under-specified, you can safely assume that it has been done intentionally. You are free to make assumptions, but please specify the assumptions you make in the report. No further clarifications on the problems will be provided. However, feel free to contact the TAs in case you believe that there exists an error in the problem specification.

1. Supervised Learning: Link
2. Deep Learning: Link

1. Tutorial 1: Naive Bayes Classifier - 08 February 2023 11:00 AM - 12:00 PM - Tony Gracious
2. Tutorial 2: Linear Models - 16 February 2023 05:00 - 06:00 PM - Parag Dutta - Link to Code
3. Extra Lecture 1: 02 March 2023 05:30 PM - Prof. Ambedkar Dukkipati
4. Problem Sheet Discussion and Demo: SVM - 16 March 2023 05:30 PM - Ranga Shaarad Ayyagari
5. Extra Lecture 2: 21 March 2023 05:30 PM - Prof. Ambedkar Dukkipati
6. Tutorial 3: Neural Networks - 25 March 2023 03:30 PM - Parag Dutta - Link to Code
7. Extra Lecture 3: 25 March 2023 05:00 PM - Prof. Ambedkar Dukkipati
8. Tutorial 4: Autograd - 02 April 2023 06:00 PM - Parag Dutta

1. Sheet 1: Bayesian Risk Minimization Link
2. Sheet 2: SVM Link

Recommended Textbooks:

• C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
• Hastie T, Tibshirani R and Friedman J, The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer, 2nd Edition, 2009
• Haykin. S, Neural Networks and Learning Systems, Prentice-Hall, 3rd Edition, 2009
• Goodfellow, Bengio, Courville, Deep Learning, MIT Press, 2017

Additional Textbooks:

• T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2nd Edition, 2009. [Free Download Available]
• T. Mitchell, Machine Learning. McGraw Hill, 1997.

Optimization Textbooks:

• Fletcher R., Practical Methods of Optimization, John Wiley, 2000.
• David G. Luenberger and Yinyu Ye, Linear and Nonlinear Programming, Springer
• Yurii Nesterov,Introductory Lectures on Convex Optimization, Springer
• Stephen Boyd,Convex Optimization, Cambridge University Press

As part of this course, you are required to work on a project. This will give you hands-on experience of working with data from various domains - text, images, videos etc that are used in the contemporary ML research, and also expose you to some specialized topics of ML that are more advanced or recent than what will be covered in the lectures. A list of project ideas will be mailed on the mailing list. You can also come up with your own projects ideas.

Project Policy:

• Some projects ideas are based on reading and survey of several papers, and others are based on implementation of one or two papers, and comparison of the methods proposed in them over datasets. We will point you to relevant papers and datasets. You can read/implement other relevant papers and datasets if you want. You can also innovate and come up with your own approaches for solving the problems.
• The best grades will be reserved for the groups which come up with some new approaches, either theoretically or experimentally, which have not been used in the literature.
• Each project will have an associated project mentor. He/she will help you to locate papers and datasets, and clear doubts.
• In implementation-based projects, it may so happen that codes are already available online. It is perfectly fine to use them. Only, that should be cited in the reports.
• You are required to provide a mid-term report around the third week of March, and the final report shortly before the final examination.
• Plagiarism is strictly prohibited while writing the reports. Any plagiarism detected will result in F grade for the course.