计算机科学与技术学院欢迎你 |

新闻通知

  • 重要新闻
  • 综合新闻
  • 讲座通知
  • 学生工作
  • 通知公告

光熙论坛(第80期)Accommodating LLM training over decentralized computational resources

来源: 作者:发布时间:2023-10-11阅读:

讲座题目:Accommodating LLM training over decentralized computational resources

讲座时间:2023年10月13日 16:00-17:30

讲座人:袁彬航助理教授

地点:H301

讲座内容:

Training algorithms for large language models are often communication heavy. As a result, these models are trained dominantly in a centralized environment such as data centers with fast network connections. This strong dependency on fast interconnections is becoming the limiting factor of further scaling for the data center setting and alternative decentralized infrastructures such as spot instances and geo-distributed volunteer computes. In this talk, I will discuss our research in communication-efficient distributed learning and our current effort in training foundation models in a decentralized way

讲座人简介:

Binhang YUAN is an Assistant Professor at the Department of Computer Science and Engineering (CSE), the Hong Kong University of Science and Technology (HKUST). He received his Ph.D. and master's degrees from Rice University and his bachelor's degree from Fudan University. Before


× 学术夏令营 招生信息