The need

When students are learning mathematics, searching for relevant material can be challenging – typing an equation with complex symbols is cumbersome, and finding learning materials such as step-by-step techniques and videos is time-consuming. Students need self-learning tools that are convenient to use on mobile devices and connected to related resources from the web.

The idea

Leveraging advances in computer vision and language understanding, we wanted to build a seamless experience that would deliver a rich mathematical learning experience to students, starting with a simple photo of the mathematics problem on their mobile devices.

The solution

Part of Microsoft Math and the Bing application, the mathematics service uses optical character recognition (OCR) to read a photo of a handwritten problem, solving the challenge of typing in complex equations. The mathematics solver engine, hosted on Azure, generates step-by-step explanations and interactive graphs. To enhance educational value, powerful Bing Search APIs provide related content, online video lectures and practice problems.

Technical details for Microsoft Math

The solution consists of a mobile experience powered by an intelligent, geo-distributed backend service hosted on Azure. The mobile experience is designed in React-Native to make the development cross-platform easier.

We built a mathematics solver backend using the same mathematics solver engine which is also integrated in OneNote. This is a compute-intensive service which is hosted on Azure cloud and geo-distributed across data centres to provide a performant mathematics experience on the client.

In addition to the solver, we developed and trained a mathematical language to understand the problem type and trigger appropriate answer modules. Using Bing Search APIs, which is available as part of Azure Cognitive Services, we can analyse the mathematical problem, and provide relevant mathematical concepts and video answers.


Projects related to Microsoft Math

Browse more innovation sandbox projects

Explore the possibilities of AI

Jump-start your own AI innovations with learning resources and development solutions from Microsoft AI.