CMPT 985 (ST): Machine Learning for Shape, Structure, and Functionality (Spring 2022)

"We have a habit in writing articles published in scientific journals to make the work as finished as possible, to cover up all the tracks, to not worry about the blind alleys or describe how you had the wrong idea first, and so on. So there isn't any place to publish, in a dignified manner, what you actually did in order to get to do the work." - Feynman, Richard Philips (Nobel Lecture)

| Course summary | Grading scheme | Lecture and presentation schedule | Review or critique | Reading lists |

General information

NameOfficePhoneEmailOffice hours
  Richard (Hao) Zhang     TASC 8027     268-6843     haoz at sfu dot ca     Tuesdays 13:00-13:50 (Zoom for now)  

  Lectures (in person):     Tuesdays 10:30-12:20 (TASC 9204 East) and Fridays 10:30 - 11:20 (TASC 9204 East)  
  Lectures (online):     Zoom link  
  Discussion forum:     Piazza  

Course summary:

In this research-oriented and seminar-based course, we will study and explore the latest research topics surrounding machine learning for 3D vision and computer graphics, with a focus on understanding, prediction, and generation of 3D shapes, their structures and functionalities. Our coverage will span the whole spectrum of how shapes are perceived and applied, from low-level representations such as point clouds, voxels, meshes, and implicit functions, to higher-level characterizations through part organizations and hierarchies, and all the way to contextual setups that reveal how objects are being used, i.e., affordance and functionality.

Classes will be held in the form of seminars, paper reading, and open discussions. Course material will be extracted from the current literature. At the end, each student will write a speculative (i.e., no implementation is requested) proposal for a new concept, algorithm, or application that is related to the covered topics.

Some topics covered:

  • Shape vs. structure vs. functionality
  • Structure-aware shape processing: structure representations, inference, and co-analyses
  • Representation learning for 3D shapes
  • Learning generative models of 3D structures
  • Affordance and functionality: how to describe, learn, and apply
  • Other topics, e.g., creative modelling, AI for arts, etc.

    Grading scheme

    Class participation and discussion (15%); two paper presentations (2 x 20% = 40%); one     paper reviews/critiques (15%); a written proposal for a future concept/design/problem/application related to one or more special topics covered in the course (30%)

    Lecture and presentation schedule

    Wk 1 - Jan 11

    • All: "Round-table" introduction
    • Richard: A Quick Course Introduction [slides]
    Wk 1 - Jan 14

    • Richard: Symmetry and Functionality [slides]
    Wk 2 - Jan 18

    • Richard: Symmetry and Functionality [slides]
    Wk 2 - Jan 21

    • Richard: 3D Shape Representations and Representation Learning for 3D Shapes [slides]
    Wk 3 - Jan 25

    • Ashish Sinha: A Planar-Reflective Symmetry Transform for 3D Shapes [paper | slides | video]
    • Maham Tanveer: Layered Analysis of Irregular Facades via Symmetry Maximization [paper | slides | video]
    • Shichong Peng: Symmetry Hierarchy of Man-Made Objects [paper | slides | video]
    Wk 3 - Jan 28

    • Richard: Partial intrinsic symmetries (slides) and multi-scale symmetries (slides)
    Wk 4 - Feb 1

    • Sanjay Haresh: What Makes a Chair a Chair? [paper | slides]
    • Perry Peng: Shape2Pose: Human-Centric Shape Analysis [paper | slides | video]
    • Ruiqi Wang: Learning to Predict Part Mobility from a Single Static Snapshot [paper | slides | video]
    Wk 4 - Feb 4

    • Aryan Mikaeili: Predictive and Generative Neural Networks for Object Functionality [paper | slides | video]
    Wk 5 - Feb 8

    Wk 5 - Feb 11

    Wk 6 - Feb 15

    • Jonas Kraasch: Modeling by Example [paper | slides]
    • Aditya Vora: Consistent Segmentation of 3D Models [paper | slides]
    • Sai Raj Kishore: Unsupervised Co-Segmentation of a Set of Shapes via Descriptor-Space Spectral Clustering [paper | slides]
    Wk 6 - Feb 18

    • Rafael Arias: Active Co-Analysis of a Set of Shapes [paper | slides]
    Wk 7 - Feb 22 & 25

    Reading week.

    Wk 8 - Mar 1

    • Hanxiao Jiang: Component-wise Controllers for Structure-Preserving Shape Manipulation [paper | slides]
    • Maham Tanveer: Photo-Inspired Model-Driven 3D Object Modeling [paper | slides]
    Wk 8 - Mar 4

    Wk 9 - Mar 8

    • Jonas Kraasch: Fit and Diverse: Set Evolution for Inspiring 3D Shape Galleries [paper | slides]
    • Rafael Arias: CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms [paper | slides]
    Wk 9 - Mar 11

    • Richard: Shape Registration and Correspondence [slides]
    Wk 10 - Mar 15

    • Sai Raj Kishore Perla PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation [paper | slides]
    • Perry Deng: Dynamic Graph CNN for Learning on Point Clouds [paper | slides]
    • Aditya Vora: Deep Closest Point: Learning Representations for Point Cloud Registration [paper | slides]
    Wk 10 - Mar 18

    • Richard: Spectral Mesh Processing [slides]
    Wk 11 - Mar 22

    • Richard: Spectral Mesh Processing [slides]
    • Ruiqi Wang: Mesh Segmentation via Spectral Embedding and Contour Analysis [paper | slides]
    Wk 11 - Mar 25

    • Hanxiao Jiang: SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation [paper | slides]
    Wk 12 - Mar 29

    • Richard: Learning Generative Models of 3D Shapes [slides]
    • Richard: Analysis and Synthesis of 3D Indoor Scenes [slides]
    Wk 12 - Apr 1

    • Richard: Unsupervised Learning of 3D Structures [slides]
    Wk 13 - Apr 5

    • Shichong Peng: Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling [paper | slides]
    • Ashish Sinha: Learning Representations and Generative Models for 3D Point Clouds [paper | slides]
    • Aryan Mikaeili: GRASS: Generative Recursive Autoencoders for Shape Structures [paper | slides]
    Wk 13 - Apr 8

    • Sanjay Haresh: GRAINS: Generative Recursive Autoencoders for INdoor Scenes [paper | slides]

    Reading lists:

    Surveys (for reference; do not pick for paper presentation)

    1. [Mitra et al. 2012] Niloy Mitra, Mark Pauly, Michael Wand, and Duygu Ceylan, Symmetry in 3D Geometry: Extraction and Applications, Eurographics State-of-the-art Report 2012.

    2. [Mitra et al. 2014] Niloy Mitra, Michael Wand, Hao Zhang, Daniel Cohen-Or, Vladmir Kim, and Qi-xing Huang, Structure-Aware Shape Processing, SIGGRAPH Course 2014 (also Eurographics State-of-the-art Report 2013).

    3. [Chaudhuri et al. 2020] Siddhartha Chaudhuri, Daniel Ritchie, Jiajun Wu, Kai Xu, and Hao Zhang, Learning Generative Models of 3D Structures, Eurographics State-of-the-art Report 2020.

    4. [Hu et al. 2018] Ruizhen Hu, Manolis Savva, and Oliver van Kaick, Functionality Representations and Applications for Shape Analysis, Eurographics State-of-the-art Report 2018.

    5. [Xie et al. 2021] Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar, Neural Fields in Visual Computing and Beyond, arXiv 2021.

    6. [Xu et al. 2016] Kai Xu, Vladimir Kim, Qixing Huang, and Evangelos Kalogerakis, Data-Driven Shape Analysis and Processing, SIGGRAPH Course 2016 (also Eurographics State-of-the-art Report 2015).

    Symmetry analysis, representation, and power of symmetry

    1. [Podolak et al. 2006] Joshua Podolak, Philip Shilane, Aleksey Golovinskiy, Szymon Rusinkiewicz, and Thomas Funkhouser, A Planar-Reflective Symmetry Transform for 3D Shapes, SIGGRAPH 2006.

    2. [Zhang et al. 2013] Hao Zhang, Kai Xu, Wei Jiang, Jinjie Lin, Daniel Cohen-Or, and Baoquan Chen, Layered Analysis of Irregular Facades via Symmetry Maximization, SIGGRAPH 2013.

    3. [Zhang et al. 2013] Yanzhen Wang, Kai Xu, Jun Li, Hao Zhang, Ariel Shamir, Ligang Liu, Zhiquan Cheng, and Yueshang Xiong, Symmetry Hierarchy of Man-Made Objects, Eurographics 2011.

    4. [Wu et al. 2020] Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi, Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild, CVPR 2020 (Best Paper Award winner).

    Structure-aware processing

    3D representation learning

    3D generative modeling

    Affordance and functionality

    1. [Grabner et al. 2011] Helmut Grabner, Juergen Gall, and Luc Van Gool, What Makes a Chair a Chair?, CVPR 2011.

    2. [Kim et al. 2014] Vladimir G. Kim, Siddhartha Chaudhuri, Leonidas Guibas, and Thomas Funkhouser, Shape2Pose: Human-Centric Shape Analysis, CVPR 2011.

    3. [Hu et al. 2017] Ruizhen Hu, Wenchao Li, Oliver van Kaick, Ariel Shamir, Hao Zhang, and Hui Huang, Learning to Predict Part Mobility from a Single Static Snapshot, CVPR 2011.

    4. [Hu et al. 2018] Ruizhen Hu, Zihao Yan, Jingwen Zhang, Oliver van Kaick, Ariel Shamir, Hao Zhang, and Hui Huang, Predictive and Generative Neural Networks for Object Functionality, CVPR 2011.

    Other topics

    School of Computing Science, Faculty of Applied Sciences, Simon Fraser University