![\"igure](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/06\/Screenshot-2025-07-08-at-1.50.40\u202fPM-1024x357.png\")
<\/figure>\n\n\n\nFigure 1. Performance comparison on representative reasoning benchmarks spanning mathematics (HMMT, AIME 25, OmniMath), scientific (GPQA), and coding (LiveCodeBench 8\/24-1\/25) domains. <\/p>\n\n\n\n
<\/p>\n\n\n\n
Notably, Phi-4-reasoning and Phi-4-reasoning-plus achieve better performance than o1-mini, and DeepSeek-R1-Distill-Llama-70B at most benchmarks and achieve performance comparable to the full DeepSeek-R1 model (with 671B parameters) on AIME 20251<\/sup> (the 2025 qualifier for the USA Math Olympiad). They also outperform Claude 3.7 Sonnet and Gemini 2 Flash Thinking on all tasks except GPQA (PhD-level STEM questions) and Calendar Planning.<\/p>\n\n\n\n More Potential with Parallel Test-time Scaling:<\/strong> As shown in Figure 2, our small-ish model nearly saturates performance on AIME 2025 with increasing parallel test-time compute (e.g., Majority @N), surpassing the pass@1 of the teacher (o3-mini). <\/p>\n\n\n\n Figure 2: Effects of parallel test-time compute on AIME 2025<\/p>\n\n\n\n <\/p>\n\n\n\n Table 1. Average Pass@1 accuracy on selected reasoning benchmarks. Bold denotes best model per benchmark and model class (open versus closed-weight), and underline denotes the second best. We report the standard deviation in parentheses where available. <\/p>\n\n\n\n Below we summarize the core contributions that led to the superior performance of Phi-4-reasoning models. We provide more comprehensive technical details and experimentations surrounding each bullet point in our tech repot [1]. <\/p>\n\n\n\n We think that reasoning is a transferable meta-skill that can be learned through supervised finetuning alone and further enhanced with reinforcement learning. To test the generalization of the models\u2019 reasoning capabilities, we evaluate them on multiple new reasoning benchmarks that require algorithmic problem solving and planning, including 3SAT (3-literal Satisfiability Problem), TSP (Traveling Salesman Problem), and BA-Calendar planning. These reasoning tasks are nominally out-of-domain for the models as the training process did not target these skills, but the models show strong generalization to these tasks as shown in Figure 2. <\/p>\n\n\n\n Table 2. Average pass@1 accuracy on general-purpose benchmarks, averaged across five runs. <\/p>\n\n\n\n This generalized improvement in capabilities also goes beyond reasoning. Without explicit training on non-reasoning tasks, we saw significant improvements on IFEval, FlenQA, and internal PhiBench as shown in Table 2. And despite limited coding data during the SFT stage (and none during RL), the model performs well, scoring at o1-mini level on LiveCodeBench (LCB) and Codeforces as shown in Table 1. We plan to emphasize coding further in our future versions.<\/p>\n\n\n\n Figure 3. Average Pass@1 performance on reasoning benchmarks, averaged across five runs. Except for GPQA, other benchmarks are out-of-distribution with respect to Phi-4-reasoning\u2019s training data.\u00a0<\/p>\n\n\n\n Language models exhibit large generation nondeterminism, i.e., they may produce substantially different answers given the same prompts and inference hyperparameters (e.g., temperature). To account for this stochastic nature, we study the accuracy distribution on AIME 2025, approximated by kernel density estimation of 50 independent runs with the same prompt and temperature. We have found several interesting observations as illustrated in Figure 4: <\/p>\n\n\n\n Figure 4. Distribution of pass@1 accuracy on AIME 2025, approximated by kernel density estimation over 50 runs with the same prompt and temperature. The accuracy distribution further shows the competitive performance of Phi-4-reasoning-plus, largely intersecting with o3-mini\u2019s distribution and being almost disjoint from the DeepSeek-R1-Distill-Llama-70B. <\/p>\n\n\n\n <\/p>\n\n\n\n Below we provide some interesting example responses from Phi-4-reasoning that showcases its intelligent behavior. <\/p>\n\n\n\n Prompt: “Generate a website for steves pc repairs using a single html script”<\/p>\n\n\n\n![\"Figure](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/07\/Screenshot-2025-07-08-at-2.07.12\u202fPM-1024x584.png\")
<\/figure>\n\n\n\n![\"Average](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/07\/Screenshot-2025-07-08-at-1.51.51\u202fPM-1024x354.png\")
<\/figure>\n\n\n\nKey contributors to best-in-class performance <\/h2>\n\n\n\n
\n
\n
\n
Reasoning is a meta skill <\/h2>\n\n\n\n
![\"Average](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/07\/Screenshot-2025-07-08-at-1.53.12\u202fPM.png\")
<\/figure>\n\n\n\n![\"Figure](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/07\/Screenshot-2025-07-08-at-2.04.33\u202fPM-1024x735.png\")
<\/figure>\n\n\n\nLessons on Evaluating Reasoning Models <\/h2>\n\n\n\n
\n
\n
\n
![\"chart,](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/07\/Screenshot-2025-07-08-at-2.03.15\u202fPM.png\")
<\/figure>\n\n\n\nPhi-4-Reasoning in action<\/h2>\n\n\n\n
![\"Example](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/06\/image-868x1024.jpeg\")
<\/figure>\n\n\n\n![\"Example](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2025\/07\/Screenshot-2025-07-08-at-2.14.03\u202fPM-890x1024.png\")
<\/figure>\n\n\n\n