Introduction to Deep Learning
Intro
This course provides a beginner’s introduction to Artificial Neural Networks (ANN) and Deep Neural Networks (DNN) and their applications to various AI tasks, including image classification, speech recognition, and natural language processing. By the end of the course, students are expected to have a significant familiarity with the subject and be able to apply deep learning to a variety of tasks.
Students will be introduced to the basic concepts and terminology associated with deep neural networks. We will start with the basics of learning in artificial neural networks and their optimizations, and we will cover the most commonly used DNN architectures such as Feed Forward Neural Networks, Convolutional Neural Networks and Recurrent Neural Networks.
We will use the Jupyter Notebook environment for coding and build our models using the PyTorch library.
Syllabus
This is the planned schedule for the whole course, but of course, changes are possible due to unforeseen reasons.
The mapping between lectures and course sessions is not one to one. Meaning that we might work on several lectures in one course session or working on one lecture for several sessions.
| Lecture # | Title | Lecture Topics | Recitations | Externals |
|---|---|---|---|---|
| 1 | Introduction |
About this course Artificial neural network: a history The perceptron Multi-layer perceptrons |
What is PyTorch? Who uses PyTorch? Why use PyTorch? PyTorch fundamentals |
|
| 2 | Feed Forward Neural Networks |
Model representation Loss functions Optimization and Backpropagation Stochastic gradient descent |
PyTorch workflow basics Computational graphs Automatic differentiation Building your first model |
|
| 3 | Training the Model I |
Data preparation Bias-Variance trade-off |
Model training in PyTorch Pytorch regression |
Homework 01 House Price Prediction |
| 4 | Training the Model II |
Hyperparameters Regularizations |
PyTorch binary classification PyTorch multi-class classification Sigmoid and Softmax |
|
| 5 | Convolutional Neural Networks |
Computer vision basics Convolutional layers Pooling layers |
PyTorch for computer vision PyTorch image classification |
Homework 02 FashionMNIST Classification |
| 6 | Recurrent Neural Networks |
Recurrent neurons Recurrent layers Long term dependencies Natural language processing basics |
PyTorch for natural language processing Embedding layers PyTorch text classification |
Homework 03 Sentiment Analysis |
| 7 | Transfer Learning |
Motivation What is transfer learning? Popular architectures |
PyTorch transfer learning Fine tuning a pretrained models Robust image classification |
Homework 04 Activity Recognition |
| 8 | In-class project | In-class project |
Saeed Varasteh Yazdi
Assistant Professor, Machine Learning Researcher