The in-house monitoring of elders using intelligent sensors is a very desirable service that has the potential of increasing autonomy and independence while minimizing the risks of living alone. Because of this promise, the efforts of building such systems have been spanning for decades, but there is still a lot of room for improvement. Driven by the recent technology advances in many of the required components, in this paper we present a scalable framework for detailed behavior interpretation of elders. We report on our early deployment experiences and present our current progress in three main components: sensors, middleware and behavior interpretation mechanisms that aim to make effective monitoring and assistive services a reality.