LLM profiling guides KV cache optimization
| Liyuan Liu and Jianfeng Gao
LLMs rely on memory-intensive mechanisms like the key-value (KV) cache to store and quickly retrieve data. FastGen optimizes KV cache usage, reducing LLM memory demands by up to 50% while maintaining performance.
LoftQ: Reimagining LLM fine-tuning with smarter initialization
| Nikos Karampatziakis, Chen Liang, Weizhu Chen, Yixiao Li, Yifan Yu, and Tuo Zhao
LoftQ boosts LLM efficiency by streamlining the fine-tuning process, reducing computational demands while preserving high performance. Innovations like this can help make AI technology more energy-efficient.
In the news | Politico
Microsoft goes from bad boy to top cop in the age of AI
Alongside efforts to use artificial intelligence to find new cures for cancer and combat climate change, the Microsoft AI for Good's small engineering team has another job: figuring out how to detect AI-powered deepfake videos, audio clips and images bombarding…
Deciding between fundamental and applied research is a dilemma that confronts many in the scientific community. Dongqi Han, on the cusp of graduation, ambitiously aspired to bridge this divide by pursuing both avenues of research in his future endeavors. After…
Abstracts: May 6, 2024
| Michel Galley and Gretchen Huizinga
Researcher Michel Galley explores how he and fellow researchers combined new and existing data to create MathVista, an open-source benchmark for measuring the mathematical reasoning capabilities of foundation models in scenarios that involve text and images.
The Appropriate Reliance research initiative focuses on advancing research and creating practical solutions for fostering appropriate reliance on AI. Through appropriate reliance (opens in new tab), we aim to help people who use AI systems find a balance between over-trusting…
Large language model-powered tools like Copilot have the potential to increase labor productivity more than any technology in a generation. Motivated by the significance of this moment, researchers from across Microsoft have come together to measure and improve the productivity…
Examining how AI can and should influence software engineering - including its effects on developers, how it improves developer efficiency, how it can assist in keeping software safe, and what the potential risks are.
Research Focus: Week of April 29, 2024
In this edition: Can LLMs transform natural language into formal method postconditions; Semantically aligned question + code generation for automated insight generation; Explaining CLIP performance disparities on blind/low vision data; plus recent news.