Esha Choukse的肖像

Esha Choukse

Principal Researcher

Connect on LinkedIn

新闻与深度文章

奖项 | ACM SIGMICRO

Choukse was recognized for her foundatio…

Research Focus | September 23, 2024
微软研究院博客

Welcome to Research Focus, a series of b…

Research Focus April 15, 2024
微软研究院博客

In this issue: New research on appropria…

An example of the generative LLM inference process and the two phases associated with it. The initial prompt is “Which is better, pizza or burger?” and it generates the word “Pizza”. The token generation phase generates the words/tokens: “is”, “better”, and “.”. The prompt phase has the following properties: (1) all input tokens are processed in parallel to generate the first output token, (2) compute intensive, and (3) is a smaller part of the end-to-end latency. The token phase is: (1) serialized, (2) memory intensive, and (3) tends to be the majority of the end-to-end latency.
微软研究院博客

| Esha Choukse, Chaojie Zhang, Íñigo Goiri, Aashaka Shah, Saeed Maleki, Rodrigo Fonseca, 和 Ricardo Bianchini

Expanded LLM use creates new demands on …