Distributed Machine Learning
Goal of the course
The goal of the mini-course is to provide a comprehensive overview of state-of-the-art of distributed machine learning. All the team members come together to learn the fundamental technical issues in depth, and discuss the possible generalizations of this field. The entire course is divided into ten sections, mainly covering the basic knowledge of machine learning, the framework of distributed machine learning, the distributed machine learning algorithms and the mainstream distributed machine learning systems.
Course Schedule
Section 1：Machine learning foundation and distributed machine learning framework——2020.07.03 Slides
Presenter：Qingyang Duan
· Basic concepts and processes of machine learning
· Common machine learning models and optimization methods
· The framework of distributed machine learning
Section 2：The deterministic algorithms of single machine optimization——2020.07.06 Slides
Presenter：Yicheng Wang
· First order deterministic algorithm
· Second order deterministic algorithm
· The dual method
Section 3：Single machine stochastic optimization algorithms——2020.07.08 Slides
Presenter：Simiao Jiao
· Basic stochastic optimization algorithm
· Improvement of stochastic optimization algorithm
· Non-convex random optimization algorithm
Section 4：Data parallelism and model parallelism——2020.07.10 Slides
Presenter：Haoyu Chen
· Computational parallel mode
· Data parallel mode
· Model parallel pattern
Section 5：Communication mechanism——2020.07.13 Slides
Presenter：Xinyu You
· The content of communication
· The topology of communication
· The pace of communication
· The frequency of communication
Section 6：Data and model aggregation——2020.07.15 Slides
Presenter：Qingyang Duan
· The aggregation method based on model sum
· Aggregation approach based on model integration
Section 7：Distributed machine learning algorithms——2020.07.17 Slides
Presenter：Simiao Jiao
· Synchronization algorithm
· Asynchronous algorithm
· Comparison and fusion of synchronous and asynchronous
· Model parallel algorithm
Section 8：Distributed machine learning theory——2020.07.20 Slides
Presenter：Ying Zheng
· Convergence analysis
· Acceleration ratio analysis
· Generalized analysis
Section 9：Distributed machine learning systems——2020.07.22 Slides
Presenter：Bowen Ding / Haoran Chen
· Spark
· Multiverso
· TensorFlow