This course explores the opportunities for High-Performance Computing (HPC) in the field of Deep Learning. By the end of the course, learners will understand the workload characteristics of deep learning models, the importance of work efficiency and speed of light in HPC, and various techniques such as Elastic SGD, optimized kernels, and memory-efficient backpropagation. The course covers skills like dense compute, fast interconnects, model parallelism, and low precision training. The teaching method includes lectures on deep learning scales, specialized systems, and potential pitfalls in ignoring efficiency and speed considerations. This course is intended for individuals interested in leveraging HPC for deep learning applications.