Deep Learning: Computing and Application

深度學習：運算與應用

Course description

Deep Learning (DL) has since the ImageNet LSVRC-2012 contest established itself as one of the core technologies for prediction with many commercial applications. The success of the current wave of Artificial Narrow Intelligence (ANI) is due to: (i) big labelled data, (ii) GPU accelerated computing, (iii) open source software, and (iv) mature algorithms. In this course, we will discuss what AI and DL is and the basic mathematical and statistical theory behind DL. You will also get a concise introduction to Python and get to apply a Neural Network to recognise digits in images, using TensorFlow/PyTorch.

自從ImageNet LSVRC-2012競賽以來，深度學習（DL）便已成為眾多商業應用中作為預測的核心技術之一。 當前的人工智慧技（ANI）的成功歸因於：（i）標記大數據，（ii）GPU加速計算，（iii）開源軟體，以及（iv）成熟的演算法。 在本課程中，我們將討論什麼是AI和DL以及DL背後的數學基礎和統計理論。學生將學習有關Python程式語言的基礎，並使用TensorFlow / PyTorch應用神經網絡來識別圖像中的數字。。

Course Objectives

  • Define Artificial Intelligence. 
  • Explain what Deep Learning is. 
  • Explain what an Artificial Neural Network is. 
  • Explain how Convolution works. 
  • Explain how Backpropagation works.
  • Explain how Gradient descent works.
  • Explain the architecture of a Deep Neural Network.
  • Read and write Python code for TensorFlow/PyTorch. 
  • Apply a Neural Network to recognise digits

Anticipated Benefits

This course will give you three things:

  1. A concise introduction to Python with the aim of bringing even students that have never programmed in Python to an intermediate level.

  2. Experience applying deep learning in practice to recognise digits using the MNIST data set.
  3. A concise introduction to the mathematical and statistical foundation of Deep Learning.

Instructor

 Ass. Prof. Torbjörn E. M. Nordling, Ph.D.

Teaching Strategies

How this course is going to be taught

  • Using online teaching 50% 50%
  • Case study 40% 40%
  • Discussion 10% 10%

Course material

Short videos explaining a topic with companion lecture notes, suggested reading, exercises, and/or example code. All material is or will be freely available online. All material produced specifically for this course will be under the Creative Commons Attribution 4.0 International (CC BY 4.0) license with attribution to Prof. Nordling, Nordling Lab.

References
  1. Goodfellow, I., Bengio, Y. & Courville, A., 2016. Deep learning, Cambridge, MA, U.S.A.: MIT Press. Available at: http://www.deeplearningbook.org.
  2. Shaw, Z., 2017. Learn Python 3 the Hard way, Addison-Wesley, ISBN 9780134692883. http://learnpythonthehardway.org/book/
  3. PEP 8 https://www.python.org/dev/peps/pep-0008/

Course policy

  1. Group project (60%) – Examined based on final presentation during a synchronized (or physical) lecture, which should contain a demonstration of how well the trained Deep Neural Network performs on an independent test data set. A Jupyter notebook with both the training and prediction code, as well as the model specification and final model (saved in HDF5 format), need to be committed to the GIT repository before the last lecture.
  2. Final exam (40%) – 
The exam will be conducted online using Google forms. The final exam will consist of questions with multiple choice.


Course outline

1. Application of Deep Learning to digit recognition using MNIST: example

i

What is AI, ML, DL, Data Science, Data Mining, Statistics, and System Identification?

i

Digit recognition using Deep Learning

i

Nordling Lab – Python and Deep learning tutorial using TensorFlow/Pytorch in Colaboratory

2. Basic programming in Python (Getting help and comments, Printing, Conditional statements and loops, Variables and functions, Reading and writing files, Error handling)

i

Basic programming in Python

3. Introduction to Neural Networks, Gradient descent, Backpropagation, and loss functions

i

Introduction to Neural Networks

3Blue1Brown – But what *is* a Neural Network? (Last visited 2021-09-30)

3Blue1Brown – Gradient descent, how neural networks… (Last visited 2021-10-09)

3Blue1Brown – What is backpropagation really doing?  (Last visited 2021-10-09)

3Blue1Brown – Backpropagation calculus (Last visited 2021-10-09)

4. Introduction to data preprocessing, model selection, regularization, activation functions, and neural network types, including convolution 

i

Introduction to Model Selection

5. Introduction to Model validation (test/training split, cross-validation, under-/overfitting, bias variance tradeoff)

i

Introduction to Model Validation

6. Application of Deep Learning to digit recognition using MNIST: project

i

Instructions for the digit recognition group project

7. Intermediate programming in Python (List comprehension, Class definition and inheritance, Modules, Lambda functions, Function decorators, Async IO, Virtual environments and PIP)

i

Intermediate programming in Python

8. Application of Deep Learning to digit recognition using MNIST: project

9. Exam and project presentation

Course websites