DATA 442: Neural Networks & Deep Learning (Spring 2021)

DATA 442: Lectures

Please note the below may change throughout the course. Any changes will be announced on Piazza.

Any video player that supports *.mkv can play the lecture files; I personally recommend VLC as it will allow you to change the speed of my voice (I intentionally talk slow!).

While this course has no assigned book, I highly recommend Deep Learning by Goodfellow, Bengio and Courville. It can be viewed for free online.

#	Topic	Materials	Other Notes
1	Course Introduction & The History of Deep Learning	Lecture Video Lecture Transcript Lecture Slides	Remember to check the Syllabus for due dates for each assignment!
2	Machine Learning in the Context of Computer Vision	Lecture Video Lecture Transcript Lecture Slides Example Code	CIFAR-10 can be seen at https://www.cs.toronto.edu/~kriz/cifar.html . We also provided a mirror here.
3	Linear Classification & Loss in Computer Vision	Lecture Video Lecture Transcript Lecture Slides
4	Softmax & Regularization	Lecture Video Lecture Transcript Lecture Slides	Chapter 7 in Deep Learning provides more information on the topic of regularization. While not required, it is recommended reading.
5	Optimization	Lecture Video Lecture Transcript Lecture Slides Example Code	Chapter 8 in Deep Learning provides more information on the topic of optimization. While not required, it is recommended reading.
6	Backpropagation (BP)	Lecture Video Lecture Transcript Lecture Slides
7	Backpropogation, Heterogeneity and Neural Nets	Lecture Video Lecture Transcript Lecture Slides
8	Convolutional Layers	Lecture Video Lecture Transcript Lecture Slides
9	Building and Optimizing a Neural Network - Part A	Lecture Video Lecture Transcript Lecture Slides	Suggested Readings: Understanding the difficulty of training deep feedforward neural networks - Glorot and Bingio, 2010 On the importance of initialization and momentum in deep learning - Sutsskever et al. 2013 How to start training: The effect of initialization and architecture - Hanin and Rolnick 2018 All you need is a good init - Mishkin and Matas 2016 Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift - Ioffe and Szegedy 2015
10	Building and Optimizing a Neural Network - Part B	Lecture Video Lecture Transcript Lecture Slides	Suggested Readings: Adam: A Method for Stochastic Optimization Practical Methods (Deep Learning Book
11	Building and Optimizing a Neural Network - Part C	Lecture Video Lecture Transcript Lecture Slides	Suggested Readings: Snapshot Ensembles: Train 1, get M for free Dropout: A Simple Way To Prevent Neural Networks from Overfitting
12	Network Architectures, Hardware and Software	Lecture Video Lecture Transcript Lecture Slides	Suggested Readings: A Performance Comparison of CUDA and OpenCL Rethinking the Inception Architecture for Computer Vision A Review of Popular Deep Learning Architectures: AlexNet, VGG16, and GoogleNet
13	Recurrent Neural Networks	Lecture Video Lecture Transcript Lecture Slides	Suggested Readings: An Analysis of Deep Neural Network Models for Practical Applications
14	Generative Networks	Lecture Video Lecture Transcript Lecture Slides	Suggested Readings: Generative Adversarial Nets (Ian Goodfellow) Pixel RNN (van der Oord) Auto-Encoding Variational Bayes Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
15	Visualizing Networks & Deep Q-Learning	Lecture Video Lecture Transcript Lecture Slides	Suggested Readings: Grad_CAM: Visual Explanations from Deep Networks vis Gradient-based Localizations