Microsoft Research Cambridge

Machine Intelligence

Advanced machine learning research, grounded in trust, efficiency, capability.

KBLaM blog | A flowchart illustrating the process of handling a prompt using a language model. The process begins with documents being used to construct and summarize a knowledge base (KB) offline. The summarized KB is then encoded and fed into the main process. A prompt goes through a tokenizer, followed by rectangular attention, and then into the large language model (LLM). The LLM retrieves information from the encoded KB to generate an answer.

Introducing KBLaM: Bringing plug-and-play external knowledge to LLMs

Layered image of Phi Silica, a state-of-the-art small language model integrated into Windows 11 Copilot+PCs

Phi Silica, small but mighty on-device SLM

A Ladder of Reasoning: Testing the power of imagination in LLMs

Diffusion

Diffusion Language Models (DLMs) explores the potential of diffusion-based models to exceed the generative quality of traditional autoregressive language models, while overcoming their current limitations in training and inference efficiency. Although early results indicate that DLMs can produce more coherent and diverse outputs than autoregressive models, these benefits have yet to be demonstrated at scale—possibly due to slower training and expensive inference. Our DLM work builds upon our recent image diffusion work where we studied the effect of the forward process in the Fourier domain, allowing us to precisely control image reconstruction across frequencies by considering the signal-to-noise of image spatial statistics. By considering similar ordered statistics in language, we hope to improve training and inference regimes in DLMs.

Learn more:

A Fourier Space Perspective on Diffusion Models
Publication | May 2025