COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Slides will be finalized after each lecture.
Event TypeDateDescriptionCourse Materials
Lecture Tuesday, Sept. 5 Intro to Deep Learning, historical context. [slides]
[python/numpy tutorial]
[jupyter tutorial]
Lecture Thursday, Sept. 7 Image classification and the data-driven approach
k-nearest neighbor
Linear classification
[slides]
[image classification notes]
[linear classification notes]
Lecture Tuesday, Sept. 12 Loss functions [slides]
Lecture Thursday, Sept. 14 Optimization: Stochastic Gradient Descent and Backpropagation [slides]
[optimization notes]
Lecture Tuesday, Sept. 19 Backpropagation & Neural Networks I [slides]
[backprop notes]
[Efficient BackProp] (optional)
related: [1], [2], [3] (optional)
Lecture Thursday, Sept. 21 Neural Networks II
Higher-level representations, image features
Vector, Matrix, and Tensor Derivatives
handout 1: Vector, Matrix, and Tensor Derivatives
handout 2: Derivatives, Backpropagation, and Vectorization
[slides]
Deep Learning [Nature] (optional)
Lecture Tuesday, Sept. 26 Neural Networks III [slides]
tips/tricks: [1], [2] (optional)
Lecture Thursday, Sept. 28 Training Neural Networks I:
Activation Functions
[slides]
[Neural Nets notes 1]
Lecture Tuesday, Oct. 3 Training Neural Networks II:
weight initialization, batch normalization
[slides]
[Neural Nets notes 2]
Lecture Thursday, Oct. 5 Training Neural Networks II (cont.):
weight initialization, batch normalization
[slides]
[Batch Norm]
No class Tuesday, Oct. 10 Monday class schedule will be followed (Columbus Day: Monday, Oct. 9)
Lecture Thursday, Oct. 12 Project announcement
Training Neural Network III:
babysitting the learning process, hyperparameter optimization
[slides]
[Neural Nets notes 3]
LeNet (optional)
Copula Normalization (optional)
Lecture Tuesday, Oct. 17 Final project information session
[slides]
[Stanford cs231n project reports]
[2016 Fall project reports]
Lecture Thursday, Oct. 19 Training Neural Network IV:
parameter updates, model ensembles, dropout
[slides]
[Bengio 2012] (optional)
Guest Lecture Tuesday, Oct. 24 Emma Strubell: An Introduction to Neural Networks for Natural Language Processing
[slides]
Lecture Thursday, Oct. 26 Convolutional Neural Networks: convolution layer, pooling layer, fully connected layer
[slides]
Lecture Tuesday, Oct. 31 Training Neural Network IV (cont.):
parameter updates, model ensembles, dropout
[slides]
Lecture Thursday, Nov. 2 Midterm review [ConvNet notes]
AlexNet (optional)
Midterm Tuesday, Nov. 7 In-class midterm [midterm review sheet]
Lecture Thursday, Nov. 9 ConvNets for spatial localization, Object detection [slides]
ResNet (optional)
Lecture Tuesday, Nov. 14 ConvNets for spatial localization, Object detection (cont.) [slides] (cont.)
FCN (optional)
Lecture Thursday, Nov. 16 Understanding and visualizing Convolutional Neural Networks
Backprop into image: Visualizations, deep dream
No class Tuesday, Nov. 21 Thanksgiving; No class.
No class Thurday, Nov. 23 Thanksgiving; No class.
Lecture Tuesday, Nov. 28 Understanding and visualizing Convolutional Neural Networks
Backprop into image: Visualizations, deep dream
[slides]
Lecture Thursday, Nov. 30 Artistic style transfer
Adversarial fooling examples
Recurrent Neural Networks (RNN)
[slides]
DL book RNN chapter (optional)
min-char-rnn, char-rnn, neuraltalk2
Lecture Tuesday, Dec. 5 Recurrent Neural Networks (RNN) (cont.)
Long Short Term Memory (LSTM) (cont.)
[slides] (cont.)
The Unreasonable Effectiveness of RNN (optional)
Understanding LSTM Networks (optional)
Lecture Thursday, Dec. 7 Training ConvNets in practice [slides]
Presentation Tuesday, Dec. 12 Poster presentations