CMPT 726: Machine Learning, Fall 2012

About the course

Machine Learning is the study of computer algorithms that improve automatically through experience. It is one of the most exciting aspects of artificial intelligence, and the basis for many of its industrial applications. It is the preferred approach for many applications that model complex functions, such as face detection (auto-focus in your digital camera), hand-written digit recognition, speech recognition, and credit card fraud detection.

This course is a 700-level grad course - it will be a lecture-style course and be taught from a textbook. The course will start from the basics, no prior experience in machine learning nor pattern recognition will be presumed. Students will gain hands-on experience with state of the art machine learning algorithms via programming assignments on real datasets and a course project.

Prerequisite

The most important prerequisite for this course is a strong mathematics background, especially in linear algebra and calculus (mainly taking derivatives for function maximization). You should have background knowledge equivalent to the following SFU courses: MATH 151, 152, 240, MACM 316, STAT 270 with an A average. It is also recommended that students have taken some of MATH 251, 252, 254, 308, 309. It will be possible to refresh your knowledge at the beginning of the course, but I don't want anyone to run from the room screaming if I say "eigenvector" or "covariance matrix". If you have questions about your readiness for this course, please ask.

Syllabus

Schedule (Tentative)

Lecture Slides.

• Introductory Examples.
• Foundational Issues.
• Directed Graphical Models: Bayes Nets and Probabilities.
• Bayes Net Parameter Learning.
  • Supplement: The Laplace Correction.
• Bayes Net Classifiers.
• Decision Tree Classifiers.
• Linear Regression.
  • Andrew Moore's Slides on Overfitting (Excerpt).
• The Bias-Variance Trade-off.
• Linear Classifiers.
• Neural Networks.
• Nearest Neighbour.
• Support Vector Machines.
• Ensembles and Boosting.
• Latent Variable Models and EM.

Assignments.

• Assignment 1, due Oct 4 at 3pm. Note: This will not be graded. Please submit it anyway so I can get feedback.
  • Assignment 1 Prep code and data
• Assignment 2. For due date please check assignment or course management system.
• Assignment 3. For due date please check assignment or course management system.
• Assignment 3 code and data
• Short guide to using svmlib by the creators.
• Page 1 of Midterm 1 with instructions.
• Assignment 4. For due date please check assignment or course management system.
• Assignment 4 code and data
• Page 1 of Final Exam with instructions.

Project.

Links

Course Management System for assignment submission and grading

Programming

Programming assignments will be done in MATLAB. MATLAB is easy to use and allows researchers to quickly develop programs for machine learning and other subjects that require many standard math functions. It provides many tools for numerical computation which are useful for implementing machine learning techniques. Students who are not familiar with MATLAB are expected to acquire basic proficiency. The following resources are helpful for learning MATLAB:

• Octave, a free software that is mostly compatible with MATLAB
• Kermit Sigmon's MATLAB Primer
• matlab.el: Emacs MATLAB mode
• Mathworks' (its creator) MATLAB Tutorial page
• MATLAB Central code repository
• Kevin Murphy (UBC) has a nice page of MATLAB tips

Learning

• VideoLectures has a large collection of ML seminars and tutorials. I recommend in particular the following two.
• Andrew Ng's machine learning course. This is an introductory course aimed at undergraduates. You may have to enrol. There is an Octave Tutorial, which is similar to Matlab.
• A higher-level introduction by Daphne Koller to statistical graphical models. You may have to enrol.
• Andrew Moore's tutorials. Introductory slides on many of the topics we discuss.
• Stephen Boyd's Convex Optimization course at Stanford (videos available)
• Useful Things to Know About Machine Learning. By Pedro Domingos.

Journals and conferences

• Journal of Machine Learning Research (JMLR)
• Neural Information Processing Systems (NIPS)
• International Conference on Machine Learning (ICML) 2009
• Uncertainty in Artificial Intelligence (UAI)
• Artificial Intelligence and Statistics (AISTATS) 2009
• IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)