Event Type | Date | Description | Course Materials |
---|---|---|---|
Lecture | Tuesday, Sept. 5 | Intro to Deep Learning, historical context. |
[slides] [python/numpy tutorial] [jupyter tutorial] |
Lecture | Thursday, Sept. 7 | Image classification and the data-driven approach k-nearest neighbor Linear classification |
[slides] [image classification notes] [linear classification notes] |
Lecture | Tuesday, Sept. 12 | Loss functions |
[slides] |
Lecture | Thursday, Sept. 14 | Optimization: Stochastic Gradient Descent and Backpropagation |
[slides] [optimization notes] |
Lecture | Tuesday, Sept. 19 | Backpropagation & Neural Networks I |
[slides] [backprop notes] [Efficient BackProp] (optional) related: [1], [2], [3] (optional) |
Lecture | Thursday, Sept. 21 |
Neural Networks II Higher-level representations, image features Vector, Matrix, and Tensor Derivatives |
handout 1: Vector, Matrix, and Tensor Derivatives handout 2: Derivatives, Backpropagation, and Vectorization [slides] Deep Learning [Nature] (optional) |
Lecture | Tuesday, Sept. 26 | Neural Networks III |
[slides] tips/tricks: [1], [2] (optional) |
Lecture | Thursday, Sept. 28 |
Training Neural Networks I: Activation Functions |
[slides] [Neural Nets notes 1] |
Lecture | Tuesday, Oct. 3 |
Training Neural Networks II: weight initialization, batch normalization |
[slides] [Neural Nets notes 2] |
Lecture | Thursday, Oct. 5 |
Training Neural Networks II (cont.): weight initialization, batch normalization |
[slides] [Batch Norm] |
No class | Tuesday, Oct. 10 | Monday class schedule will be followed (Columbus Day: Monday, Oct. 9) | |
Lecture | Thursday, Oct. 12 |
Project announcement Training Neural Network III: babysitting the learning process, hyperparameter optimization |
[slides] [Neural Nets notes 3] LeNet (optional) Copula Normalization (optional) |
Lecture | Tuesday, Oct. 17 |
Final project information session |
[slides] [Stanford cs231n project reports] [2016 Fall project reports] |
Lecture | Thursday, Oct. 19 |
Training Neural Network IV: parameter updates, model ensembles, dropout |
[slides] [Bengio 2012] (optional) |
Guest Lecture | Tuesday, Oct. 24 |
Emma Strubell: An Introduction to Neural Networks for Natural Language Processing |
[slides] |
Lecture | Thursday, Oct. 26 |
Convolutional Neural Networks: convolution layer, pooling layer, fully connected layer |
[slides] |
Lecture | Tuesday, Oct. 31 |
Training Neural Network IV (cont.): parameter updates, model ensembles, dropout |
[slides] |
Lecture | Thursday, Nov. 2 | Midterm review |
[ConvNet notes] AlexNet (optional) |
Midterm | Tuesday, Nov. 7 | In-class midterm | [midterm review sheet] |
Lecture | Thursday, Nov. 9 | ConvNets for spatial localization, Object detection |
[slides] ResNet (optional) |
Lecture | Tuesday, Nov. 14 | ConvNets for spatial localization, Object detection (cont.) |
[slides] (cont.) FCN (optional) |
Lecture | Thursday, Nov. 16 |
Understanding and visualizing Convolutional Neural Networks Backprop into image: Visualizations, deep dream |
|
No class | Tuesday, Nov. 21 | Thanksgiving; No class. | |
No class | Thurday, Nov. 23 | Thanksgiving; No class. | |
Lecture | Tuesday, Nov. 28 |
Understanding and visualizing Convolutional Neural Networks Backprop into image: Visualizations, deep dream |
[slides] |
Lecture | Thursday, Nov. 30 |
Artistic style transfer Adversarial fooling examples Recurrent Neural Networks (RNN) |
[slides] DL book RNN chapter (optional) min-char-rnn, char-rnn, neuraltalk2 |
Lecture | Tuesday, Dec. 5 |
Recurrent Neural Networks (RNN) (cont.) Long Short Term Memory (LSTM) (cont.) |
[slides] (cont.) The Unreasonable Effectiveness of RNN (optional) Understanding LSTM Networks (optional) |
Lecture | Thursday, Dec. 7 | Training ConvNets in practice |
[slides] |
Presentation | Tuesday, Dec. 12 | Poster presentations |