Harish Kamath is a Computer Science/Math undergraduate at Georgia Tech. His passions focus on reinforcement learning, generalization in learning, and making newer technologies cheaper, faster, and more accessible. Outside of work, I love playing/watching basketball, dance, and running! Once I graduate, I hope I end up somewhere where I can make the biggest lasting impact for the most people.
Challenge: Conversion of VowpalWabbit models into ONNX format
Currently, ONNX is the leading standard to represent machine learning models across platforms and frameworks. It describes a model as a computational graph consisting of a set of standard operators from an operator set that is constantly evolving to accommodate new types of models and operations. Being able to describe a model in ONNX format is important, as it allows for (1) models to be optimized and run across different architectures using a single runtime, and (2) it allows models created in different frameworks to interact with each other. Although other leading frameworks such as Tensorflow and Pytorch have mature tools to convert into ONNX format, VowpalWabbit today does not yet have the capability baked into the framework. This project focuses on introducing this functionality to VowpalWabbit, so that we can combine the fast model training and inference speed of VW with the representational capacity of other frameworks. We introduce new sparse operators that are used to instantiate VW regression models efficiently in ONNX format, show that you can directly translate regression and contextual bandits models with these operators, and give an example of such models being run in RLClientLib to show that they can now be ported into any inference framework.
Cassandra Marcussen is a junior at Columbia University studying Mathematics and Computer Science. Her interests lie in artificial intelligence, theoretical computer science, and contributing to technology within these fields through efficient computing and low-level optimizations. Cassandra is enthusiastic about open source code, and has loved working on an impactful open source system such as Vowpal Wabbit. In the future, she wishes to pursue graduate studies in Computer Science.
Challenge: Parallelized Parsing
Modern machines often utilize many threads to achieve good performance. Currently, VW uses a single thread to read in and parse input, and a single thread to learn. The parse thread presents a bottleneck, slowing down VW as a whole. By extracting the input reading into a separate thread and extending the parser to support many threads, VW can better utilize resources, achieve better performance, and have an improved design by separating logical components into independent modules. This project focuses on improving performance and design for the text input format, and also ensures compatibility with the cache input format.
Newton Mwai Kinyanjui
Newton Mwai Kinyanjui is from Nairobi Kenya. I’m currently pursuing my Ph.D. at Chalmers University of Technology in Sweden, working in causal inference and reinforcement learning towards machine learning for improved decision making in healthcare with Fredrik Johansson. I graduated from Carnegie Mellon University Africa with a Master of Science in Electrical and Computer Engineering.
Challenge: Library of contextual bandit estimators
Estimators are used in off-policy evaluation. One common estimator is IPS, and others are DR and PseudoInverse. These estimators work better or worse in different settings. This project explores reference implementations of each and allows for comparison between them to aid in understanding. We extend the estimators library and implement an interface to help researchers and data scientists test different estimators quickly and easily.
Milind Agarwal is a combined undergraduate and master’s student in Computer Science at Johns Hopkins University. His current research interests are natural language processing and machine translation for low-resource and endangered language settings. Before interning at Microsoft, he previously worked in many different academic research labs at Johns Hopkins gaining experience in a wide variety of fields including NLP, machine translation, computational biology, data visualization, and software development. After graduation in 2021, he hopes to join a Ph.D. program where he can continue to work on challenging NLP problems.
Challenge: Contextual Bandit Data Visualization with Jupyter Notebooks
Exploratory data analysis and data visualization have become an essential part of any data scientist’s toolkit. Visualizations not only allow you to kickstart your analysis by easily understanding the patterns in your data but also help you visually inspect your policies to understand their behaviour. We present cb_visualize, a python-based visualization library specialized for contextual bandits features and policy visualizations. This library offers robust visualizations for data exploration, training, feature importance, and action distributions and supports common contextual-bandit dataset formats used by Vowpal Wabbit like text, JSON, and DSJSON. We hope that this toolkit will be an asset for researchers and customers alike to better present and understand their data and analyses.
Mark Rucker is currently a 2nd year PhD student at the University of Virginia with a previous 8 year career as an enterprise software engineer. Mark’s PhD research explores how reinforcement learning models can be used to encourage health behavior change in individuals managing chronic health conditions. This research combines state of the art machine learning with web and mobile app development to support in-situ randomized control trials of behavior change interventions. After graduation Mark hopes to once again return to industry in order to develop high-quality products that deeply impact people’s lives.
Challenge: COBA: A Modern Benchmarking Package for Reproducible Contextual Bandit Research
Performance benchmarking on well-defined problems is a pillar of modern machine learning research. With clear problems and metrics, benchmarking has allowed the research community to maintain a high-level of independent effort while still making real and meaningful progress over time. The elegance of benchmarking — define, measure, repeat — however, belies real engineering challenges such as software maintenance, data distribution, statistical aggregation, and reproducibility to name a few. These challenges are especially salient in contextual bandit research where one not only needs a data set but also a harness to emulate interaction with the data. In an effort to reduce these burdens, while not losing any of benchmarking’s benefits, we present COBA, an ultra light-weight Python package for benchmarking contextual bandit algorithms. COBA uses a small set of clean and consistent interfaces to satisfy four core use cases: (1) creating reproducible benchmarks, (2) sharing reproducible benchmarks, (3) evaluating custom algorithms, and (4) exploring evaluation results.
Sharad Chitlangia is a senior year undergraduate student at BITS Pilani Goa, where I studied Electronics. I am specializing in the field of Artificial Intelligence. I’ve previously worked heavily at the intersection of Machine Learning and Systems and Explainable AI. Aside from work, I spend a lot of time in the Open Source Community and working on improving accessibility, especially in AI research.
Challenge: Pushing the Limits of VW with Flatbuffers
VowpalWabbit is known for its abilitiy to solve complex machine learning problems extremely fast. Through this project, we aim to take this ability, even further, by the introduction of Flatbuffers. Flatbuffers is an efficient cross-platform serialization library known for its memory access efficiency and speed. We develop Flatbuffer schemas, for input examples, to be able to store them as binary buffers and show a performance increase of 30%, or more compared to traditional formats.