CS 431 : Programming Parallel Systems

Fall 2016
Instructor Arrvindh Shriraman
Office : Surrey: SURR 4134, Burnaby: TASC 1 9241
Email :

Instructor office hours: Fridays (14:30 - 15:30pm starting Jan 15th)
T.A : Email : nvedula@sfu.ca (or Google groups email: ) TA office hours: Wednesdays (13:30-14:30). TA will be at TASC 1 8208. Starting Jan 15th.

What is the class about?

This course explores the paradigms of parallel and distributed computing, their applications, and the systems/architectures supporting them. This course is a hands-on, programming-heavy course. Expect to get down and get your hands dirty with concurrent C++ programming. We will discuss the fundamental design and engineering trade-offs in parallel systems at every level. We will study not only how they work today, but also try to understand the design decisions and how they are likely to evolve in the future. We will draw examples from real-world parallel systems in this course.

In parallel computing, multiple processors compute sub-tasks simultaneously so that work can be completed faster. Early parallel computing has focused on solving large computation-intensive problems like scientific simulations. With the increasingly available commodity multiprocessors (like multicores), parallel processing has penetrated into many areas of general computing. In distributed computing, a set of autonomous computers are coordinated to achieve unified goals. More interestingly the community is increasingly turning to parallel programming models (e.g., Google's Mapreduce, Microsoft's Dryad Linq) to harness the capabilities of these large scale distributed systems. We will begin with a primer on multicore manycore chips: chips that integrate multiple CPUs on the same die. Currently, Intel is shipping 10 core (20 thread chips) and our own AMD servers are 2 socket 24 thread systems. Harnessing these new compute engines is an open challenge. We will focus on their architectures and different programming models that empower us to tap into this fountain of CPUs. We will studyr synchronization, cache coherence, and memory consistency. We will look at programming abstractions such as locks, mutexes, conditional synchronization, and transactional memory. At this stage we will also be exposed to the manycore accelerators such as gaming processors and GPUs which promise 100x performance improvement for "data parallel" workloads.

We will examine the parallel computing models like MPI and Mapreduce customized to distributed systems. Finally, we will perform case studies of practical distributed systems and devote significant attention to cluster-based server systems in large Internet data centers (e.g., those running Google services).