Towards Glimpses of a New Science of Brains, Minds and Machines:
Weaving Together Physics, Computer Science, and Neurobiology
Our neural circuits exploit the laws of physics to perform computations in ways that are fundamentally different from traditional computers designed by these same neural circuits. To eradicate this irony, we must develop a new science of brains, minds and machines that seamlessly weaves together physics, computation and neurobiology to both elucidate the design principles governing neural systems, and instantiate these principles in physical devices. We will discuss several glimpses in such a direction, including: (1) understanding the speed with which both infants and deep neural circuits learn hierarchical structure, (2) exploiting the geometry of high dimensional error surfaces to speed up learning, (3) exploiting ideas from non-equilibrium statistical mechanics to circumvent credit-assignment and mixing time problems to learn very deep stochastic generative models, and (4) delineating fundamental theoretical limits on the energy, speed and accuracy of communication by any physically implementable device.