Programming Models and Systems Design for Deep Learning

  • Tianqi Chen, Junyuan Xie | University of Washington

We have witnessed emergence of many deep learning systems; each comes with its own unique features. While most system will evolve and getting better, there are some fundamental design choices behind the system that will affect the how far this system can get at in terms of flexibility and performance. In this talk, I will discuss the design of deep learning system from this perspective. Specifically, I am going to talk about declarative (symbolic) and imperative programming for deep learning models and the advantage/disadvantages of each approach. I will motivate the usage of mixed design, which results in mxnet – our system that support mixed declarative and imperative programming to achieve maximum flexibility and performance. I will also talk about systems for automatic task scheduling and memory optimization for the mixed programming model.

Series: Microsoft Research Talks