Microsoft Research Blog

Splitwise improves GPU usage by splitting LLM inference phases

January 4, 2024
Expanded LLM use creates new demands on cloud GPU capacity. Splitwise presents an efficient solution by separating the two essential phases of LLM inference, achieving higher throughput within a limited power budget.
  1. Research Focus 31

    Research Focus: Week of December 18, 2023 

    December 20, 2023

    In this issue of Research Focus: Optimized exit-augmented models for scalable efficient inference; NeurIPS LLM Efficiency Challenge; LLM-empowered automated data exploration; Boosting cloud efficiency with data-driven decision-making and optimization.

  2. three conversation bubbles on a blue, purple, and pink gradient background

    Steering at the Frontier: Extending the Power of Prompting 

    December 12, 2023 | Eric Horvitz, Harsha Nori, and Yin Tat Lee

    We’re seeing exciting capabilities of frontier foundation models, including intriguing powers of abstraction, generalization, and composition across numerous areas of knowledge and expertise. Even seasoned AI researchers have been impressed with the ability to steer the models with straightforward, zero-shot prompts. Beyond basic, out-of-the-box prompting, we’ve been…

  3. Research Focus Edition 30 December 6, 2023

    Research Focus: Week of December 4, 2023 

    December 6, 2023

    Research Focus: Using LLMs in a Rust-based formal verification framework; Rethinking network measurements with user feedback; 3D telemedicine using HoloportationTM communication technology could enhance overseas surgical visits.

  4. Blue to green gradient. Two rows of hands: the top row signing ASL and the bottom row signing Data.

    Tackling sign language data inequity 

    December 4, 2023

    ASL Citizen is the first crowdsourced sign language dataset, advancing the state of the art in sign recognition. The web-based project captured input from people in real-world settings, and from a diverse group of experts, including Deaf team members.

Explore More

Events & conferences

Events & conferences 

Meet our community of researchers, learn about exciting research topics, and grow your network

Podcasts

Podcasts 

Ongoing conversations at the cutting edge of research

Microsoft Research Forum

Microsoft Research Forum 

Join us for a continuous exchange of ideas about research in the era of general AI