{"id":1145758,"date":"2025-07-24T15:07:58","date_gmt":"2025-07-24T22:07:58","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/"},"modified":"2025-07-31T11:06:25","modified_gmt":"2025-07-31T18:06:25","slug":"office-ai-science-team","status":"publish","type":"msr-group","link":"https:\/\/www.microsoft.com\/en-us\/research\/group\/office-ai-science-team\/","title":{"rendered":"Office AI Science Team"},"content":{"rendered":"
\n\t
\n\t\t
\n\t\t\t\"Stylized\t\t<\/div>\n\t\t\n\t\t
\n\t\t\t\n\t\t\t
\n\t\t\t\t\n\t\t\t\t
\n\t\t\t\t\t\n\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n

Office AI Science<\/h1>\n\n\n\n

Office AI Science builds systems that are leveraged across M365,and especially within Word, Excel, and PowerPoint<\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n

The\u00a0Office AI Science\u00a0team is\u00a0part of OPG.\u00a0We build\u00a0systems\u00a0that\u00a0are\u00a0leveraged\u00a0across\u00a0M365 and especially within Word, Excel, and PowerPoint.\u00a0The team\u2019s\u00a0recent\u00a0projects have included:\u00a0PPT\u00a0Summarization,\u00a0Audio\u00a0Overviews\u00a0(Podcast),\u00a0SPOCK Eval, Data Pipeline,\u00a0Natural\u00a0Language to\u00a0Office JS,\u00a0and\u00a0CUA.\u00a0<\/p>\n\n\n\n

PPT\u00a0Summarization:<\/em>\u00a0The Office AI Science team\u00a0built the first\u00a0fine-tuned SLM within M365.\u00a0The\u00a0fine-tuned\u00a0Phi-3 Vision SLM\u00a0improved p95 latency\u00a0of PPT Visual Summary feature\u00a0from 13 seconds to 2 seconds, while\u00a0maintaining quality (opens in new tab)<\/span><\/a>\u00a0on par with GPT-4o-v.\u00a0This\u00a0optimization resulted in\u00a075 times\u00a0fewer\u00a0GPUs being used\u00a0compared to GPT-4o-v\u00a0and almost\u00a09 times the number of\u00a0PowerPoint\u00a0users\u00a0receiving\u00a0a\u00a0visual summary.\u00a0The fine-tuned SLM also powers PPT Visual Q&A, making it both faster and cheaper.\u00a0The\u00a0team also introduced\u00a0PPT Interactive Summary, which allows users to drill into visual summaries in more detail, leading to\u00a0over\u00a050% decline in thumbs down per 100k tries\u00a0over 3 months, 30% interactivity clicking on chevron to go deeper, and a 17.6% increase in weekly return rate.\u00a0The team is currently fine-tuning\u00a04o-mini-vision\u00a0with the goal of replacing remaining non-English traffic to GPT-4o-v with this smaller model\u00a0and evaluating\u00a0Phi-4\u00a0Vision for English.\u00a0<\/p>\n\n\n\n

Audio Overviews:<\/em> The team is building the Audio Overview Skill that introduces a podcast-like experience for consuming documents and artifacts. The feature is currently in the dogfood phase for MSIT, with production rollout scheduled for May 7 onwards. Users will be able to generate Audio Overviews from App Chat entry points in Word Win32 & Web, Copilot Notebooks (including OneNote), and other apps like Outlook Web, OneDrive Web and ODSP Mobile. Latest human evaluation (opens in new tab)<\/span><\/a> scores overall transcript quality for the single file audio overview at 4.08\/5.00 compared to 3.76\/5.00 for NotebookLM, and with automated evaluation (opens in new tab)<\/span><\/a>, the team improved the overall score from an initial 4.09 to 4.65 with a two-step design leveraging GPT-4o and o3-mini. More details, including evaluation against multiple files for the Copilot Notebooks scenario and gains from moving to GPT-4.1, can be found here (opens in new tab)<\/span><\/a>. <\/p>\n\n\n\n

SPOCK (AugLoop Eval)<\/em>: In collaboration with AugLoop, the Office AI Science team developed several key features that enable agility in evaluating App Copilot scenario quality metrics. By the end of FY25Q3, 22 scenarios have been onboarded across Word, PPT, Office AI, and SharePoint, with Excel onboarding in-progress. The platform currently reliably runs 300 eval jobs and 30,000 tests daily. The automated scenario evaluation turnaround time compared to manual run has significantly decreased from days to 2-4 hours. SPOCK now supports intent detection, Leo Metrics, BizChat 1K Query, Python, and Typescript customer evaluators; model swap and FlexV3 eval are coming in Q4. Additionally, the v-team is automating the App Copilot Quality Dashboard (\u00c6VAL – Copilot Evaluation (opens in new tab)<\/span><\/a>), providing a comprehensive overview of the quality of App Copilot scenarios. <\/p>\n\n\n\n

Data Pipeline:<\/em> The team also created an online, self-serve, on-demand ADF pipeline for mining Office documents from the internet. This allows partners to kick off large-scale data mining jobs for specific languages and document types and features custom metadata extractors for extracting task-dependent document representations. By leveraging Bing\u2019s precrawled 40B URL RetroIndex, document discovery is fast and efficient. OAI Science and several partner teams (Word+Editor, PPT Science, Word Designer, Designer, MSAI) are already utilizing the data for finetuning and test set creation. <\/p>\n\n\n\n

Natural Language to Office JS:<\/em> The Office AI Science team is working to finetune o* family model for common Office scenarios like inserting slides from another PowerPoint file, inserting headers and footers in Word, or creating and finding merged ranges in Excel. <\/p>\n\n\n\n

CUA:<\/em> The team also recently embarked on an exploration of Computer User Agent (CUA) centered on understanding user intent and adapting in real time. Leveraging plan assistance with the Office knowledge base, the team approximately doubled the task completion rate against OSWorld PPT scenarios. The team is working on fine-tuning the CUA model to improve task completions for Office apps. <\/p>\n\n\n\n

For more contact: <\/em>Amanda Gunnemo<\/em><\/a> or <\/em>Vishal Chowdhary<\/em><\/a><\/em> <\/p>\n\n\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Office AI Science builds systems that are leveraged across M365,and especially within Word, Excel, and PowerPoint The\u00a0Office AI Science\u00a0team is\u00a0part of OPG.\u00a0We build\u00a0systems\u00a0that\u00a0are\u00a0leveraged\u00a0across\u00a0M365 and especially within Word, Excel, and PowerPoint.\u00a0The team\u2019s\u00a0recent\u00a0projects have included:\u00a0PPT\u00a0Summarization,\u00a0Audio\u00a0Overviews\u00a0(Podcast),\u00a0SPOCK Eval, Data Pipeline,\u00a0Natural\u00a0Language to\u00a0Office JS,\u00a0and\u00a0CUA.\u00a0 PPT\u00a0Summarization:\u00a0The Office AI Science team\u00a0built the first\u00a0fine-tuned SLM within M365.\u00a0The\u00a0fine-tuned\u00a0Phi-3 Vision SLM\u00a0improved p95 latency\u00a0of PPT Visual Summary […]<\/p>\n","protected":false},"featured_media":1145543,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_group_start":"","footnotes":""},"research-area":[13556],"msr-group-type":[243694],"msr-locale":[268875],"msr-impact-theme":[],"class_list":["post-1145758","msr-group","type-msr-group","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-group-type-group","msr-locale-en_us"],"msr_group_start":"","msr_detailed_description":"","msr_further_details":"","msr_hero_images":[],"msr_research_lab":[],"related-researchers":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-projects":[],"related-events":[],"related-opportunities":[],"related-posts":[],"tab-content":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group\/1145758","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-group"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group\/1145758\/revisions"}],"predecessor-version":[{"id":1146267,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group\/1145758\/revisions\/1146267"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1145543"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1145758"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1145758"},{"taxonomy":"msr-group-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-group-type?post=1145758"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1145758"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1145758"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}