In this paper, we explore the use of a projection-based handheld device to facilitate in-clinic doctor-patient communication. We present the user-centered design process used to understand the workflow of medical professionals and to identify challenges they currently face in communicating information to patients. Based on the lessons learned, we developed AnatOnMe, a prototype projection-based handheld system for enhancing information exchange in the current practice of one medical sub-specialty, physical therapy. We then present the results of a controlled experiment to understand the desirability and learning tradeoffs of using AnatOnMe to teach medical concepts on three potential projection surfaces – wall, model, and patient body. Finally, we present results of two expert reviews of the system.