News & features
PlugMem: Transforming raw agent interactions into reusable knowledge
| Ke Yang, Michel Galley, Chenglong Wang, Jianfeng Gao, Jiawei Han, and ChengXiang Zhai
It seems counterintuitive: giving AI agents more memory can make them less effective. As interaction logs accumulate, they grow large, fill with irrelevant content, and become increasingly difficult to use. More memory means that agents must search through larger volumes of…
Multimodal reinforcement learning with agentic verifier for AI agents
| Reuben Tan, Baolin Peng, Zhengyuan Yang, Oier Mees, and Jianfeng Gao
Argos improves multimodal RL by evaluating whether an agent’s reasoning aligns with what it observes over time. The approach reduces visual hallucinations and produces more reliable, data-efficient agents for real-world applications.
MindJourney enables AI to explore simulated 3D worlds to improve spatial interpretation
| Yuncong Yang, Reuben Tan, Swadheen Shukla, and Jianfeng Gao
MindJourney can enable AI to navigate and interpret 3D environments from limited visual input, potentially improving performance in navigation, planning, and safety-critical tasks.
CollabLLM: Teaching LLMs to collaborate with users
| Shirley Wu, Michel Galley, Baolin Peng, Swadheen Shukla, and Jianfeng Gao
Recipient of an ICML 2025 Outstanding Paper Award, CollabLLM improves how LLMs collaborate with users, including knowing when to ask questions and how to adapt tone and communication style to different situations. This approach helps move AI toward more user-centric…
Research Focus: Week of April 21, 2025
In this issue: our CHI 2025 & ICLR 2025 contributions, plus research on causal reasoning & LLMs; countering LLM jailbreak attacks; and how people use AI vs. AI-alone. Also, SVP of Microsoft Health Jim Weinstein talks rural healthcare innovation.
Research Focus: Week of March 24, 2025
In this issue, we examine a new conversation segmentation method that delivers more coherent and personalized agent conversation, and we review efforts to improve MLLMs’ understanding of geologic maps. Check out the latest research and other updates.
Magma: A foundation model for multimodal AI agents across digital and physical worlds
| Swadheen Shukla, Jianwei Yang, Reuben Tan, Qianhui Wu, and Jianfeng Gao
Explore Magma, a foundation model that can empower AI assistants to interpret environments, plan actions, and execute tasks across digital and physical spaces. Now available, learn how it advances the field of agentic AI.
ExACT: Improving AI agents’ decision-making via test-time compute scaling
| Baolin Peng, Xiao Yu, Hao Cheng, Michel Galley, Zhou Yu, and Jianfeng Gao
ExACT combines Reflective-MCTS and Exploratory Learning to improve AI agents' decision-making, enabling test-time compute scaling. Learn how these methods help agents refine strategies for state-of-the-art performance and improved computational efficiency.
Data Formulator: Exploring how AI can help analysts create rich data visualizations
| Chenglong Wang, Steven Drucker, Dan Marshall, Jeevana Priya Inala, Kori Inkpen, and Jianfeng Gao
Data Formulator investigates combining UI interactions with natural language input. Powered by AI, it can help users create or adapt visualizations and supports continuous refinement through an iterative process. Now available on GitHub.