Advanced Deep Neural Networks
Course Description
| Name: Advanced Deep Neural Networks |
Duration: 5 weeks (20h of lectures, 8 h of seminars) |
|---|---|
| Lead instructor: Dr Nicholas Lane |
Price: 5450 GBP |
Key features:
Course style:
| Outcomes:
Upcoming classes:
|
Deep learning changed the world as we know it, it is the foundation on which most contemporary AI applications rest. This research-centric and code-driven advanced course will help you gain a practical, hands-on, knowledge of the fundamental concepts that made the field the focal point of advanced analytics. The course will provide you with knowledge of both, the fundamental concepts, as well as the state-of-the-art developments in the field of Deep Learning and their code-level implementation. Students will be led to develop the skills necessary to understand, implement, and research deep learning models. In the process students will develop their very own deep learning programming portfolio.
All students who complete the course will receive a course completion certificate issued by the Cambridge AI Academy, and signed by Nicholas Lane. Furthermore, students will be eligible to receive a personalized letter of performance written by Dr Nicholas Lane, suitable for use as support for applications to further study. This letter will comment on the students demonstrated abilities, skills, and progress made during the course.
Please note, none of the courses we offer have a relationship to the University of Cambridge or any other university. Cambridge AI Academy courses are taught by our academic faculty in their capacity as individuals.
Instructor - Nicholas D. Lane
Associate Professor at the Department of Computer Science and Technology at the University of Cambridge where he leads the Machine Learning Systems Lab. Prior to joining Cambridge, Dr Lane was an Associate Professor at the University of Oxford (2017 to 2020) and Senior Lecturer at the University of London (2016 to 2017). Nicholas also has more than 10 years of experience in industrial research. Alongside his academic position, he is currently a Director at the Samsung AI Centre in Cambridge. Previously, he has been a Principal Scientist at Nokia Bell Labs and a Lead Researcher at Microsoft Research in Beijing.
Course structure
This is an intensive and demanding commitment that includes 20 hours of lectures and 10 hours of TA seminars. These are taught over course of the first 4 weeks. Each week will consist of:
Two lecturer-led 2.5-hour classes will be held on Fridays and on Saturdays from 6:00 PM to 8:30 PM (China Standard Time). These are interactive undergraduate-level lectures taught by the course lead, Dr Lane, and other Cambridge AI Academy faculty including Filip Svoboda. The Friday lectures cover the core theory of Deep Learning and may be shared with the DNN class. The Saturday lectures take a deep dive into the academic as well as the industrial state of the art in deep learning.
TA-led 2.5-hour computing labs will be held on Sundays from 6:00 PM to 8:30 PM (China Standard Time). Students will be given weekly programming problem sets. The labs will introduce these exercises, demonstrate the core skills needed to complete the problem sets, and then provide the official solutions. Furthermore, these sessions will offer the students the opportunity to ask implementation questions about the problem sets and to revisit the content taught by the lecturers.
The course content is designed, and its delivery is overseen by the course lead, Nicholas Lane. He is responsible for the quality of the course delivery and its assessment. The course is delivered by the course lead, in collaboration with the Cambridge AI Academy faculty including Filip Svoboda and others.
There will be an online exam in the final week, which will test student’s achieved progress. This, alongside the student’s in-class performance in week 4 will form the basis of their evaluation.
Course syllabus
Lecture 0 (sample lecture) will introduce the background behind what made Deep Learning the juggernaut it is today and behind the forces that are likely to shape its evolution in the future. It is free and held two weeks before the primary course material begins.
Lecture 1 will introduce the core building blocks that power deep models. Both traditional architectures, as well as convolutional and recurrent networks, will be covered. Applications in vision and natural language processing will be discussed.
Lecture 2 will focus on the attention mechanism, and its use in the state-of-the-art transformer deep model architectures. The operator’s use and implementation will be motivated by and derived from a critical evaluation of standard deep learning layers.
Lecture 3 will examine the art of training deep neural networks. First, a solid understanding of deep learning parameter learning will be derived from core statistical concepts. Then, a step-by-step guide to successfully training a deep model will be built on top of this foundation.
Lecture 4 will take a deep dive into the distributed learning used to train modern industry-size deep models. A solid understanding of the state-of-the-art distributed training algorithms will be derived from a close examination of the distributed system hardware organization and its inefficiencies.
Lecture 5 will explore the fundamentals of deep learning computational resource management. The chief impediment to further deep learning adoption is its dependence on expensive large-scale compute. This lecture will tackle this dependence both from the vantage point of computer systems as well as the algorithms themselves.
Lecture 6 will fully automate the learning process by devising evolutionary and reinforcement learning agents built to propose and train deep learning models. These learning machines will learn how to learn. They will automate the otherwise labour-intensive model building stage in deep learning and do it in a way that will achieve the best performance given the user’s computational budget.
Lecture 7 will be structured as a NeurIPS-style student-led research conference chaired by Dr Nicholas Lane. The students will learn how to effectively present their work and to convince their peers about the importance of their research. They will be asked to each present a paper assigned to them in a research seminar setting that will simulate a high-profile industry-academic conference.
Lecture 8 will take a deep dive into the hottest topics in the bleeding-edge deep learning literature. The most exciting and influential deep learning papers will be presented, and their methods will be analysed in detail. The students will learn how to critically assess and learn from the best of the best papers in the deep learning literature.
The course will conclude with a multiple-choice exam conducted in real-time online in week 5 and a 10-minute personalized one-to-one feedback session in week 6.
Students will be provided with a certificate of accomplishment and will be able to request letters of recommendation based on their in-course performance for up to 24 months after finishing the course.