Effective reversible (aka time travel) debugging of arbitrary native code


September 22, 2016


Greg Law


Undo Software


UndoDB is a reversible debugger based on a proprietary JIT binary instrumentation implementation. It runs on Linux and Android user-mode processes, on 32-bit and 64-bit x86 and ARM. It is integrated with gdb’s reversible debugging meaning that users can step or run backwards as well as forwards; reverse execution with watchpoints is particularly compelling. We also have a library that can be embedded in a program and allow it to record itself in production or in test-suites for subsequent offline analysis. UndoDB uniquely works with almost any unmodified executable (including those that use multiple threads, asyncronous IO, shared memory, non-determistic instructions, and even RDMA), on almost any Linux kernel (as long as it’s 2.6 or later), on almost any x86 or ARM CPU. Its performance makes it practical for some very demanding workloads: our customers including Credit Suisse, JPMorgan, Synopsys, Cadence, IBM and LG use it daily with workloads of up to 100’s of GB’s working set, with run times of days or even weeks. Users quickly become dependent on the functionality; one recently described it as “like heroine”. In this talk we will give a demo and present some of the technical details of the implementation, and then discuss some exciting potential future applications of deterministic replay, beyond simply debugging. We will also discuss our experiences of using the technology in the real world, and some of the challenges (including cultural ones) that we have encountered in building and rolling out our tool and building our company.