The proliferation of connected sensing devices (or Internet of Things) can in theory enable a range of “smart” applications that make rich inferences about users and their environment. But in practice, developing such applications today is arduous because they are constructed as monolithic silos, tightly coupled to sensing devices, and must implement all data sensing and inference logic, even as devices move or are temporarily disconnected. We present Beam, a framework and runtime for distributed inference-driven applications that breaks down application silos by decoupling their inference logic from other functionality. It simplifies applications by letting them specify “what should be sensed or inferred,” without worrying about “how it is sensed or inferred.” We discuss the challenges and opportunities in building such an inference framework.