982/499 : Special Topics in Computer Architecture and Systems : Machine Learning Fall 2019
What is the class about ? Machine learning - Hardware
The advancements and overwhelming success of Machine Learning has profoundly affected the future of computer architecture. Not only is performing learning on big-data the leading application driver for future architectures, but also machine learning techniques can be used to improve hardware efficiency for a wide variety of application domains.
This course will explore, from a computer architecture perspective, the principles of hardware/software codesign for machine learning. One thrust of the course will delve into accelerator, CPU, and GPU enhancements for ML algorithms, including parallelization techniques. The other thrust of the course will focus on how machine learning can be used to optimize conventional architectures by dynamically learning and adapting to program behavior.
IS THIS AN ML COURSE?
Not really – the computation behind machine learning and how that is exploited with hardware is what is most relevant here. That said, I welcome ML experts, or even projects that focus on the algorithm aspect of a problem (provided there is some relationship to hardware or hardware support).
Course Objectives
The advancements and overwhelming success of Machine Learning has profoundly affected the future of computer architecture. Not only is performing learning on big-data the leading application driver for future architectures, but also machine learning techniques can be used to improve hardware efficiency for a wide variety of application domains.
This course will explore, from a computer architecture perspective, the principles of hardware/software co-design for machine learning. One thrust of the course will delve into accelerator, CPU, and GPU enhancements for ML algorithms, including parallelization techniques. The other thrust of the course will focus on how machine learning can be used to optimize conventional architectures by dynamically learning and adapting to program behavior.
Several important specific goals are:
Develop skills in domain-specialization (reason about how application/domain properties can be exploited with hardware mechanisms) Gain understanding of the current state of the art within acceleration for machine learning, both in academia and in industry. Also, there are some general goals which hold for any architecture/hardware course:
Gain intuition and reasoning skills regarding fundamental architecture tradeoffs of hardware design choices (performance/area/power/complexity/generality). Understand microarchitecture techniques for extracting parallelism and exploiting locality. Learn about evaluation methods, including simulation, analytical modeling, and mechanistic models.
Course Components:
Logistically, this course has 4 components.
- Online discussion on google groups: During this course we will read a number of research papers from literature. We will discuss these on google groups. See review page for more instructions.Critique
- In-class discussion: You should aim to participate in every in-class discussion. I will do my best to track who is participating. Remember, this class will be fun and interesting if you make it so! Each student (or group of 2 students) will lead 1 lecture.
- Mini-Projects (15%) There will be two mini projects which can also be performed in groups: Build a ML-accelerators, which is correct and produces accurate performance estimates.
- Project: Project (40%) Group based research/implementation project with 2-4 students. Please see the project handout, and feel free to use Piazza to help form groups. You will need to propose a project by the beginning of the 5th week of class, so please start thinking early. See project page for more details.
Prereq
CMPT 225 and CS295. Solid programming skills (C++). Expert at Unix/Linux. If unsure whether you have the pre-requisites, talk to the instructor! Programming exercises will require the student to implement programs illustrating the principles used in OS design. Moderate knowledge of C and C++ programming and the UNIX environment is assumed.
For homeworks and programming assignments, students are encouraged to consult each other, the TA, the instructor, or anyone else for that matter. However, the assistance offered or accepted should not go beyond a discussion of the problem and a sketch of a solution. You can use the following guideline: when it comes time for you to write your program or your homework paper, do not use any written material from the discussion.
If you can reconstruct the discussion and complete the solution on your own, then you have learned the material (and that is the objective of this course!). For team projects, you should make sure to identify division of labor in your README. While projects will generally be graded as a team rather than separately for each individual, corrective action could be taken.
Instructor
Teaching Assistant (please read FAQ page for email etiquette)
Office hours: Mondays.
Name: Parmida Vahdanthia@sfu.ca Emails: pvahdantia@sfu.ca
Name: Reza Hojabr@sfu.ca Emails: shojabro@sfu.ca
Discussion Board
Google groups (preferred medium)
Time and place
AQ 5016 Burnaby Monday and Wed 14:30-15:20. Fridays (No class; project discussion only. Instructors’s office.) Sep 3, 2019 to Dec 10, 2019
Textbook
No required textbook. Online readings provided in schedule page.
Grading
Submit homework source code and check your grades on Coursys
- THREE programming homeworks. (40%)
- Class presentation and participation (20%)
- Project: (40%)