The current information explosion fundamentally changes how people live and work with computing: vast numbers of documents and images are available on the Web; ubiquitous sensing enables near-continuous tracking and monitoring of people and objects; and inexpensive storage allows people to keep near-unlimited personal data and sensing archives. One strategy to enable effective access to and interaction with such large unstructured datasets is to support example-based iterative end-user training of machine learning systems to identify relevant concepts. These concepts can then be used specify desired manipulations. In the context of the CueFlik system for re-ranking Web image search results according to their visual characteristics, we have been examining general questions surrounding the design of end-user interactive machine learning.