Despite advances in Artificial Intelligence, fully autonomous intelligent systems are still out of the reach of current technology. Recently however, crowdsourcing has been shown to be an effective means of completing tasks AI alone cannot yet handle. This talk will describe recent work on real-time continuous crowdsourcing systems powered by the “Crowd Agent” framework, which we introduced to enable human intelligence to power intelligent systems capable of interacting with users in real-world settings. I will also describe how it is possible to use Crowd Agents as a scaffold for deployable AI systems, taking advantage of the strengths of both human and machine contributors. I will then discuss my work at MSR on potential privacy and security threats to interactive crowd-powered systems, and how privacy can be maintained by pre-filtering content using the crowd itself.