Three Small Steps … to Reconceiving Machine Learning

November 8, 2010
Prof. Bob Williamson | Australian National University

I will show by way of three separate illustrations my work on my long term project of reconceiving machine learning. After a brief introduction to the project, I will show a new and explicit characterisation of the convexity of proper composite losses (the composition of a proper binary loss or scoring rule with a link function). Such losses are the appropriate choice for binary class probability estimation. Second I will show an apparently novel relationship between M-estimators (where one maximises an objective function) and L-estimators (linear combinations of order statistics). Finally I will merely sketch an intriguing connection between the design of loss functions for prediction problems and different uncertainty calculi that have been developed in the economics literature. Intriguingly, there are results that show that even if one starts from a pure “Bayesian” perspective, one is inexorably lead to nonlinear expectations that do not fit within the framework of probability theory. The conclusion is that to do a proper job of being the “new science of uncertainty” machine learning needs to look well beyond the theory of probability.

Speaker Details

Robert C. Williamson is Professor and Head of the Computer Sciences Laboratory at the Australian National University. He is also Scientific Director of NICTA, Australia`s Centre of Excellence in Information and Communication Technology. His current research is in Machine Learning.
Previously he has worked in Signal Processing. He received his PhD in Electrical Engineering from the University of Queensland.