Speech recognition of names in Personal Information Management (PIM) systems is an important yet difficult task. The difficulty arises from various sources: the large number of possible names that users may speak, different ways a person may be referred to, ambiguity when only first names are used, and mismatched pronunciations. In this paper we present our recent work on name recognition with User Modeling (UM), i.e., automatic modeling of user’s behavior patterns. We show that UM and our learning algorithm lead to significant improvement in the perplexity, Out Of Vocabulary rate, recognition speed, and accuracy of the top recognized candidate. The use of an exponential window reduces the perplexity by more than 30%.