Observing and Controlling Distributed Systems with Cross-Cutting Tools

  • Jonathan Mace | Brown University

Distributed systems represent some of the most interesting and successful computing applications in use today, from modern web applications and social networks, to data analytics and large-scale machine learning. However, it is notoriously difficult to understand, troubleshoot, and enforce distributed systems behaviors, because unlike standalone programs, they lack a central point of visibility and control. This impacts a range of important tasks, such as resource management, performance, security, accounting, and many more. In this talk, I will outline several techniques and abstractions that let us re-establish this missing cross-component visibility and control. To demonstrate, I will present two cross-cutting tools that I have developed in my research: Retro, which measures resource usage and co-ordinates scheduler parameters to achieve end-to-end performance goals; and Pivot Tracing, which dynamically monitors and correlates metrics across component boundaries. Together, these tools illustrate some of the common challenges and potential solutions when developing and deploying tools for distributed systems.

Speaker Details

Jonathan Mace is a Ph.D. candidate in the Computer Science department at Brown University advised by Professor Rodrigo Fonseca. His research centers on how to understand and enforce end-to-end behaviors in distributed systems. During his Ph.D. he was awarded the Facebook Fellowship in Distributed Systems, and he received an SOSP Best Paper Award for his work on Pivot Tracing. Jonathan received his undergraduate degree in Mathematics and Computer Science from Oxford University in 2009.

Watch Next