How well do Computers Solve Math Word Problems? Large-Scale Dataset Construction and Evaluation

Meeting of the Association for Computational Linguistics |

Published by Association for Computational Linguistics

PDF | Publication | Publication | Publication

Download BibTex

Recently a few systems for automatically solving math word problems have reported promising results. However, the datasets used for evaluation have limitations in both scale and diversity. In this paper, we build a large-scale dataset which is more than 9 times the size of previous ones, and contains many more problem types. Problems in the dataset are semi-automatically obtained from community question-answering (CQA) web pages. A ranking SVM model is trained to automatically extract problem answers from the answer text provided by CQA users, which significantly reduces human annotation cost. Experiments conducted on the new dataset lead to interesting and surprising results.

Related Tools

SigmaDolphin

Building a computer system to automatically solve math word problems written in natural language. SigmaDolphin is a project initiated in early 2013 at Microsoft Research Asia, with the primary goal of building a computer intelligent system with natural language understanding and reasoning capacities. We focus on the application of automatic problem solving, i.e., automatically solving problems (especially math word problems) written in natural language. The dataset can also be downloaded from the Microsoft Download Center: https://www.microsoft.com/en-us/download/details.aspx?id=105300

Access