There is increasing interest in using computing applications towards the socio-economic development of the poor. However, because poverty commonly correlates with illiteracy, researchers have identified various usability challenges that low-literate users may encounter in interacting with traditional text-based UIs. To counter such problems, researchers have proposed non-textual UIs or these users. However, most current work focuses exclusively on illiteracy (the inability to read) per se, with little recognition to other problems or the overall context in which a user is situated. In this paper we suggest that the inability to read is only one of several possible concerns that prevent useful interaction of existing computing (PC and mobile phone) UIs by low-literate users. Through our ethnographic and usability studies with 400 low-literate, low-income subjects across India, the Philippines and South Africa, we find a host of nuanced issues which mediate how a user interacts with computing technologies. Such issues include: cognitive difficulties, collaboration, cultural etiquette, experience and exposure, intimidation, mediation, motivation, pricing, power relations, social standing, and others. We observe that these factors can have far-reaching influence on the design of UIs as well as services for low-literate populations.