In this paper I present a new approach to designing virtual machines. I argue that the monolithic structure of existing systems leads to unnecessarily rigid high-level abstractions being imposed on applications. In contrast, I propose an extensible approach within which key parts of the virtual machine can be updated independently of one another. Furthermore, the proposed approach allows untrusted application code to replace low-level components of the virtual machine, such as the thread scheduler or memory allocation subsystem.