The need

When students are learning math, searching for relevant material can be challenging - typing an equation with complex symbols is cumbersome, and finding learning materials such as step-by-step techniques and videos is time-consuming. Students need self-learning tools that are convenient to use on mobile devices and connected to related resources from the web.

The idea

Leveraging advances in computer vision and language understanding, we wanted to build a seamless experience that would deliver a rich math learning experience to students, starting with a simple photo of the math problem on their mobile devices.

The solution

Part of Microsoft Math and the Bing application, the math service uses optical character recognition (OCR) to read a photo of a handwritten problem, solving the challenge of typing in complex equations. The math solver engine, hosted on Azure, generates step-by-step explanations and interactive graphs. To enhance educational value, powerful Bing Search APIs provide related content, online video lectures, and practice problems.

Technical details for Microsoft Math

The solution consists of a mobile experience powered by an intelligent, geo-distributed backend service hosted on Azure. The mobile experience is designed in React-Native to make the development cross-platform easier.

We built a math solver backend using the same math solver engine which is also integrated in OneNote. This is a compute-intensive service which is hosted on Azure cloud and geo-distributed across data centers to provide a performant math experience on the client.

In addition to the solver, we developed and trained a math language to understand the problem type and trigger appropriate answer modules. Using Bing Search APIs, which is available as part of Azure Cognitive Services, we can analyze the math problem, and provide relevant math concepts and video answers.

Resources:

Projects related to Microsoft Math

Browse more innovation sandbox projects

Explore the possibilities of AI

Jumpstart your own AI innovations with learning resources and development solutions from Microsoft AI.