Predictive user models often require a phase of effortful supervised training where cases are tagged with labels that represent the status of unobservable variables. We formulate and study principles of lifelong learning where training is ongoing over a prolonged period. In lifelong learning, decisions are made continuously about the value of probing users for the values of unobservable states associated with different situations. The learner continually weighs the cost of interruption of probes for unobservable states with the long-term benefits of acquiring the new label. We highlight key principles by extending Busybody, an application that learns to predict the cost of interrupting a user in different settings. We transform the prior BusyBody system into a lifelong learner and then review experiments that show the value of the methods.