Buildings account for a large portion of the world’s total delivered energy consumption. With smartphones becoming increasingly ubiquitous and sensor networks growing more mature, buildings can provide personalized and context-aware services to an occupant while minimizing energy consumption. This paper proposes the architecture of a Human-Building-Computer Interaction system that connects the building to its occupants by bridging the gap between the digital and physical worlds. We present our instantiation of an HBCI system, which is composed of an Android mobile application, a number of RESTful services in the cloud, and physical objects co-located with QR tags. We show that, with this system, a user can increase personal comfort within a building while reducing energy usage.