{"id":1152051,"date":"2025-10-21T09:00:00","date_gmt":"2025-10-21T16:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=1152051"},"modified":"2025-11-26T14:37:35","modified_gmt":"2025-11-26T22:37:35","slug":"tell-me-when-building-agents-that-can-wait-monitor-and-act","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/tell-me-when-building-agents-that-can-wait-monitor-and-act\/","title":{"rendered":"Tell me when: Building agents that can wait, monitor, and act"},"content":{"rendered":"\n
\"Workflow<\/figure>\n\n\n\n

Modern LLM Agents can debug code, analyze spreadsheets, and book complex travel. Given those capabilities, it\u2019s reasonable to assume that they could handle something simpler: waiting. Ask an agent to monitor your email for a colleague\u2019s response or watch for a price drop over several days, and it will fail. Not because it can\u2019t check email or scrape prices. It can do both. It fails because it doesn\u2019t know when<\/em> to check. Agents either give up after a few attempts or burn through their context window, checking obsessively. Neither work. <\/p>\n\n\n\n

This matters because monitoring tasks are everywhere. We track emails for specific information, watch news feeds for updates, and monitor prices for sales. Automating these tasks would save hours, but current agents aren\u2019t built for patience.<\/p>\n\n\n\n

To address this, we are introducing SentinelStep (opens in new tab)<\/span><\/a>, a mechanism that enables agents to complete long-running monitoring tasks. The approach is simple. SentinelStep wraps the agent in a workflow with dynamic polling and careful context management. This enables the agent to monitor conditions for hours or days without getting sidetracked. We’ve implemented SentinelStep in Magentic-UI<\/a>, our research prototype agentic system, to enable users to build agents for long-running tasks, whether they involve web browsing, coding, or external tools. <\/p>\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tSpotlight: AI-POWERED EXPERIENCE<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"\"\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

Microsoft research copilot experience<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Discover more about research at Microsoft through our AI-powered experience<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tStart now\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

How it works<\/h2>\n\n\n\n

The core challenge is polling frequency. Poll too often, and tokens get wasted. Poll too infrequently, and the user\u2019s notification gets delayed. SentinelStep makes an educated guess at the polling interval based on the task at hand\u2014checking email gets different treatment than monitoring quarterly earnings\u2014then dynamically adjusts based on observed behavior. <\/p>\n\n\n\n

There\u2019s a second challenge: context overflow. Because monitoring tasks can run for days, context overflow becomes inevitable. SentinelStep handles this by saving the agent state after the first check, then using that state for each subsequent check.<\/p>\n\n\n\n

\n
These demonstrations capture Magentic-UI with SentinelStep at work, completing a range of tasks in a timelapse sequence. <\/figcaption><\/figure>\n\n\n\n

Core components<\/h3>\n\n\n\n

As the name suggests, SentinelStep consists of individual steps taken as part of an agent\u2019s broader workflow. As illustrated in Figure 1, there are three main components: the actions necessary to collect information, the condition that determines when the task is complete, and the polling interval that determines timing. Once these components are identified, the system\u2019s behavior is simple: every [polling interval] <\/em>do [actions] <\/em>until [condition] <\/em>is satisfied.<\/em> <\/p>\n\n\n\n

\"Figure\u00a01.\u00a0SentinelSteps\u2019s\u00a0three
Figure 1. SentinelSteps\u2019s three main components in Magentic-UI\u2019s co-planning interface. <\/em><\/figcaption><\/figure>\n\n\n\n

These three components are defined and exposed in the co-planning interface of Magentic-UI. Given a user prompt, Magentic-UI proposes a complete multi-step plan, including pre-filled parameters for any monitoring steps. Users can accept the plan or adjust as needed.<\/p>\n\n\n\n

Processing<\/h3>\n\n\n\n

Once a run starts, Magentic-UI assigns the most appropriate agent from a team of agents to perform each action. This team includes agents capable of web surfing, code execution, and calling arbitrary MCP servers.<\/p>\n\n\n\n

When the workflow reaches a monitoring step, the flow is straightforward. The assigned agent collects the necessary information through the actions described in the plan. The Magentic-UI orchestrator then checks whether the condition is satisfied. If it is, the SentinelStep is complete, and the orchestrator moves to the next step. If not, the orchestrator determines the timestamp for the next check and resets the agent\u2019s state to prevent context overflow.<\/p>\n\n\n\n

Evaluation<\/h2>\n\n\n\n

Evaluating monitoring tasks in real-world settings is nearly impossible. Consider a simple example: monitoring the Magentic-UI repository on GitHub until it reaches 10,000 stars (a measure of how many people have bookmarked it). That event occurs only once and can\u2019t be repeated. Most real-world monitoring tasks share this limitation, making systematic bench marking very challenging.<\/p>\n\n\n\n

In response, we are developing SentinelBench, a suite of synthetic web environments for evaluating monitoring tasks. These environments make experiments repeatable. SentinelBench currently supports 28 configurable scenarios, each allowing the user to schedule exactly when a target event should occur. It includes setups like GitHub Watcher, which simulates a repository accumulating stars over time; Teams Monitor, which models incoming messages, some urgent; and Flight Monitor, which replicates evolving flight-availability dynamics. <\/p>\n\n\n\n

Initial tests show clear benefits. As shown in Figure 2, success rates remain high for short tasks (30 sec and 1 min) regardless of whether SentinelStep is used. For longer tasks, SentinelStep markedly improves reliability: at 1 hour, task reliability rises from 5.6% without SentinelStep to 33.3% with it; and at 2 hours, it rises from 5.6% to 38.9%. These gains demonstrate that SentinelStep effectively addresses the challenge of maintaining performance over extended durations.<\/a><\/p>\n\n\n\n

\"Figure
Figure 2. SentinelStep improves success rates on longer running tasks (1\u20132 hours) while maintaining comparable performance on shorter tasks.  <\/figcaption><\/figure>\n\n\n\n

Impact and availability<\/h2>\n\n\n\n

SentinelStep is a first step toward practical, proactive, longer\u2011running agents. By embedding patience into plans, agents can responsibly monitor conditions and act when it matters\u2014staying proactive without wasting resources. This lays the groundwork for always\u2011on assistants that stay efficient, respectful of limits, and aligned with user intent.<\/p>\n\n\n\n

We\u2019ve open-sourced SentinelStep as part of Magentic-UI, available on GitHub (opens in new tab)<\/span><\/a> or via pip install magnetic-ui<\/code>. As with any new technique, production deployment should be preceded through testing and validation for the specific use case. For guidance on intended use, privacy considerations, and safety guidelines, see the Magentic-UI Transparency Note. (opens in new tab)<\/span><\/a> <\/p>\n\n\n\n

Our goal is to make it easier to implement agents that can handle long-running monitoring tasks and lay the groundwork for systems that anticipate, adapt, and evolve to meet real-world needs. <\/p>\n","protected":false},"excerpt":{"rendered":"

SentinelStep enables AI agents to handle monitoring tasks that run for hours or days, like watching for emails or tracking prices. It works by managing when agents should check and their context, avoiding wasted resources and missed updates.<\/p>\n","protected":false},"author":43518,"featured_media":1152522,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_hide_image_in_river":null,"footnotes":""},"categories":[1],"tags":[],"research-area":[13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[269148,243984,269142,269145],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-1152051","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-post-option-approved-for-river","msr-post-option-blog-homepage-featured","msr-post-option-include-in-river","msr-post-option-pinned-for-river"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[992148],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[],"related-researchers":[{"type":"user_nicename","value":"Hussein Mozannar","user_id":43671,"display_name":"Hussein Mozannar","author_link":"Hussein Mozannar<\/a>","is_active":false,"last_first":"Mozannar, Hussein","people_section":0,"alias":"hmozannar"},{"type":"guest","value":"matheus-kunzler-maldaner","user_id":"1152615","display_name":"Matheus Kunzler Maldaner","author_link":"Matheus Kunzler Maldaner<\/a>","is_active":true,"last_first":"Maldaner, Matheus Kunzler","people_section":0,"alias":"matheus-kunzler-maldaner"},{"type":"user_nicename","value":"Maya Murad","user_id":43879,"display_name":"Maya Murad","author_link":"Maya Murad<\/a>","is_active":false,"last_first":"Murad, Maya","people_section":0,"alias":"mayamurad"},{"type":"guest","value":"jingya-chen","user_id":"767776","display_name":"Jingya Chen","author_link":"Jingya Chen<\/a>","is_active":true,"last_first":"Chen, Jingya","people_section":0,"alias":"jingya-chen"},{"type":"user_nicename","value":"Gagan Bansal","user_id":41707,"display_name":"Gagan Bansal","author_link":"Gagan Bansal<\/a>","is_active":false,"last_first":"Bansal, Gagan","people_section":0,"alias":"gaganbansal"},{"type":"user_nicename","value":"Rafah Hosn","user_id":36783,"display_name":"Rafah Hosn","author_link":"Rafah Hosn<\/a>","is_active":false,"last_first":"Hosn, Rafah","people_section":0,"alias":"raaboulh"},{"type":"user_nicename","value":"Adam Fourney","user_id":30820,"display_name":"Adam Fourney","author_link":"Adam Fourney<\/a>","is_active":false,"last_first":"Fourney, Adam","people_section":0,"alias":"adamfo"}],"msr_type":"Post","featured_image_thumbnail":"\"Workflow","byline":"","formattedDate":"October 21, 2025","formattedExcerpt":"SentinelStep enables AI agents to handle monitoring tasks that run for hours or days, like watching for emails or tracking prices. It works by managing when agents should check and their context, avoiding wasted resources and missed updates.","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1152051","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/43518"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=1152051"}],"version-history":[{"count":41,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1152051\/revisions"}],"predecessor-version":[{"id":1152821,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/1152051\/revisions\/1152821"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1152522"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1152051"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=1152051"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=1152051"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1152051"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=1152051"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=1152051"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1152051"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1152051"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1152051"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=1152051"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=1152051"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}