Accelerating Machine Learning Inference with Probabilistic Predicates

Yao Lu, Aakanksha Chowdhery, Srikanth Kandula, Surajit Chaudhuri

Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD 2018) |

Published by ACM SIGMOD

View Publication

Classic query optimization techniques, including predicate pushdown, are of limited use for machine learning inference queries, because the user-defined functions (UDFs) which extract relational columns from unstructured inputs are often very expensive; query predicates will remain stuck behind these UDFs if they happen to
require relational columns that are generated by the UDFs. In this work, we demonstrate constructing and applying probabilistic predicates to filter data blobs that do not satisfy the query predicate; such filtering is parametrized to different target accuracies. Furthermore, to support complex predicates and to avoid per-query training, we augment a cost-based query optimizer to choose plans with appropriate combinations of simpler probabilistic predicates. Experiments with several machine learning workloads on a big-data cluster show that query processing improves by as much as 10×.