Image of peppers growing on a vine.

Project Gecko

As generative AI transforms productivity and access to knowledge, its benefits must extend beyond English-speaking and Western-centric contexts. Historically, each industrial revolution has introduced equity gaps profoundly impacting societal divides. In this fourth industrial revolution driven by AI, there is a valid concern that these disparities could widen further. The challenge lies in how this technology navigates the complexities of non-English languages and cultures. However, at Microsoft Research we are committed to building AI models and copilots that are equitable by design, capable of serving diverse linguistic and cultural contexts, and focused on closing, not widening, global opportunity gaps.

Project Gecko is a cross-lab Microsoft Research initiative advancing the Equitable Generative AI Project by developing equitable models and trusted multilingual copilots—designed for population scale use that can serve entire communities.

Co-led by the Microsoft Research Accelerator, Microsoft Research India, and Microsoft Research Africa, Nairobi, Gecko explores how AI can be adapted and deployed widely in low-resource settings. The project integrates research in multilingual small and large language models, multilingual speech models, synthetic data creation for low resource settings, HCI and grounded evaluation. To do this we deploy a blend of NLP, ethnographic design, and machine learning methods to create new multidisciplinary paradigms for building human-centered AI— with real-world deployments spanning agriculture and education.

Core research areas:

  • Multimodal grounding using trusted community videos
  • Speech-first interfaces for oral-first language users
  • Local adaptation of LLMs, SLMs, and speech models to support a range of languages (including Kiswahili, Hindi, Kikuyu) and cultural contexts in India and East Africa
  • Synthetic data creation for low resource settings
  • Human-centered evaluation frameworks for trust, relevance, and cultural alignment
  • Ethnomethodologically-informed research and design to co-create AI experiences with local users

Initial focus:

The initial phase of Project Gecko centers on agriculture in East Africa and South Asia, where we are investigating how trusted, multilingual Copilots can operate at population scale. In partnership with Microsoft Research Africa, Nairobi, Microsoft Research India, and Digital Green (opens in new tab) — a global development organization that partners with governments and grassroots organizations to build community-driven digital infrastructure for agriculture — we are exploring the infrastructure, modeling strategies (particularly around small language models), and evaluation protocols necessary to enable scaled deployment.

This milestone involves Farmer.Chat (opens in new tab), an AI-powered web app assistant developed by Digital Green that enables smallholder farmers to engage with community-contributed agricultural video content via a speech-first interface tailored for use in Kiswahili and Kikuyu.

Read the story

Initial pilots in Kenya demonstrate measurable improvements in response quality, usability, and user trust—offering early signals for how community-grounded, multilingual Copilots might perform in similar contexts.

Through this effort, we are also evaluating how the VeLLM platform can serve as a replicable playbook for grounded Copilot development. By analyzing what works in this agricultural context, we aim to identify generalizable design patterns, tools, and infrastructure that can be extended to future domains such as education and health — helping to enable scalable, locally relevant Copilots across a wide range of linguistic and cultural settings.

Key outputs:

  • Multi-modal Critical Thinking Agent (MMCTAgent): Enables structured reasoning over long-form video and image data, available on GitHub (opens in new tab) and featured on Azure AI Foundry Labs (opens in new tab).
  • PazaBench (opens in new tab): The first automatic speech recognition (ASR) leaderboard dedicated to low‑resource languages, it covers 61 African Languages across 53 State-of-the-Art ASR and Language Models.
  • Paza Models (opens in new tab): The Paza ASR models consist of three fine-tuned ASR models for Swahili, Dholuo, Kalenjin, Kikuyu, Maasai, and Somali built on top of state‑of‑the‑art model architectures. The models are fine-tuned on public & curated proprietary datasets, early versions of two models in Kikuyu and Swahili were tested directly with farmers in real‑world settings, enabling the team to observe how the models performed for everyday use.
  • Paza Playbook (opens in new tab): A practical guide for building Automatic Speech Recognition (ASR) models that work in the real world. The Paza Playbook helps you design models that thrive in low-resource and multilingual environments where challenges like data scarcity, linguistic diversity, and computational limits are common.
  • Vibhasha Playbook (opens in new tab): A comprehensive guide for building language model applications in multilingual and multicultural settings. Vibhasha shares frameworks, methods, and resources for making strategic decisions about multilingual AI features, localization strategies, and inclusive technology design.
  • Vibhasha Benchmark (opens in new tab): A multilingual evaluation benchmark survey with a systematic, data-driven audit of 51 multilingual benchmark suites, comprising 242 unique datasets across 220 languages.
  • Atlas Playbook (opens in new tab): A toolkit for designing, deploying, and evaluating AI across diverse cultural contexts. It draws on our learnings on Project Gecko to guide teams on how to build AI systems that fit real conditions, respect lived experience, and support the communities they serve.