Role of Simulation in Computer Vision  ICCV 2017

Role of Simulation in Computer Vision ICCV 2017



Recent progress in building intelligent systems have revealed great opportunities in use of computer vision based methods. Examples of such intelligent systems include autonomous vehicles that need depth perception and Visual Question Answering systems that need to solve scene understanding. However, many of the state-of-the-art approaches that use techniques such deep learning and reinforcement learning need large amounts of training data. There have been promising approaches that show that it might be feasible to generate artificial data via simulators in order to augment the training set. In addition, recent advance in graphics and hardware has enabled development of engines that are not only photorealistic but also run in real-time, enabling rapid training and testing of models. While such ideas are promising, there are several research challenges that still need to be addressed. For example, first is the question of how can we build a simulator that can indeed generate data that is realistic and can be useful to solve vision tasks. Secondly, techniques such as reinforcement learnings have been very useful in solving gaming tasks such as AlphaGo, Atari Games etc. it is still not clear how such techniques can enable training of systems that are deployed in real-world. The role of building and effectively using realistic simulators can be critical in bridging such simulation to reality gap.

Goals and Topics of Interest

The proposed workshop will bring together researchers from computer vision, machine learning and robotics to examine the challenges and opportunities in using simulators to build real-world vision systems. One of the key goals of the workshop is to survey state-of-the-art approaches, identify potential new directions and facilitate increased dialog between researchers within this field and the greater computer vision community. In summary, the topics we intend to cover include:

  • Enabling Photorealistic Simulation via advances in graphics and hardware
  • Simulators and Autonomous Systems
  • Synthetic Data generation for Semantic Labeling
  • Reinforcement and Imitation Learning on Simulated Data
  • Transfer Learning from Simulated Environment to Real-World
  • Enabling Deep Machine Learning with Synthetic Data


Day: October 23 (Monday)

1400 Introduction
1410 Invited Talk 1 Abhinav Gupta
1430 Invited Talk 2 Daniel Cremers
1450 Invited Talk 3 Debadeepta Dey
1510 Invited Talk 4 Devi Parikh
1530 Invited Talk 5 Pushmeet Kohli
1550 Afternoon Break
1630 Invited Talk 6 Vladlen Koltun
1650 Invited Talk 7 Davide Scaramuzza
1710 Invited Talk 8 Alan Yuille
1730 Invited Talk 9 TBD
1750 Concluding Remarks

Workshop Organizers

Sanja Fidler, Assistant Professor, University of Toronto

Ashish Kapoor, Principal Researcher, Microsoft Research

Raquel Urtasun, Associate Professor, University of Toronto

Antonio Torralba, Professor, Massachusetts Institute of Technology