{"id":557487,"date":"2018-12-18T09:52:19","date_gmt":"2018-12-18T17:52:19","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=557487"},"modified":"2018-12-18T14:01:44","modified_gmt":"2018-12-18T22:01:44","slug":"competition-win-a-steppingstone-in-the-greater-journey-to-create-sustainable-farming","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/competition-win-a-steppingstone-in-the-greater-journey-to-create-sustainable-farming\/","title":{"rendered":"Competition win a steppingstone in the greater journey to create sustainable farming"},"content":{"rendered":"
\"From

From left to right: Thomas Follender Grossfeld, Kenneth Tran, Chetan Bansal, and David Katzin of Team Sonoma<\/p><\/div>\n

The cucumber plants, their leaves wide and green and veiny, stood tall in neat rows, basking in the Netherland sunlight shining through the glass panes of their greenhouses. Hopes were high for the plants\u2014a bountiful crop in just four months using as few resources as possible. With the right amount and type of care, they\u2019d produce vegetables for consumers to enjoy. To the casual observer, though, it might have seemed like the plants had been left to their own devices. Greenhouse staff passed through to harvest or adjust cameras and other electrical devices inside, but human contact from those responsible for determining how much water, nutrition, and light the plants received was nonexistent. That was the point.<\/p>\n

This spring, Wageningen University & Research and corporate sponsor Tencent challenged researchers, scientists, and experts from across sectors: Build the greenhouse of the future. Motivated by potential strain on traditional methods of food production as a result of a growing world population and seeing a solution in greenhouses that are operational sans on-site human expertise, organizers asked competition participants to use artificial intelligence to maximize cucumber production while minimizing resources\u2014and to do so remotely.<\/p>\n

Nine months later, Microsoft Research\u2019s Team Sonoma, led by Principal Research Engineer Kenneth Tran<\/a>, beat out four other interdisciplinary teams to win the Autonomous Greenhouse Challenge (opens in new tab)<\/span><\/a>, creating an agent that produced more than 55 kilograms of cucumber per square meter with a net profit of \u20ac25\/m2<\/sup>.<\/p>\n

\u201cThis was the first time worldwide cucumbers were grown in greenhouses remotely on AI,\u201d says challenge coordinator Silke Hemming. \u201cWe at Wageningen University & Research were excited to collaborate with different teams on this exciting international challenge. Team Sonoma was able to beat the manual-grown reference operated by good Dutch growers. They not only reached the highest net profit, but the jury also ranked them highest on total sustainability.\u201d<\/p>\n

With a net profit 17 percent higher, Sonoma was the only AI team to best the reference expert growers, and its net profit was 25 percent higher than that of the second-place team, led by researchers at Tencent AI Lab. Net profit counted the most toward overall performance in the competition while algorithm novelty and capacity accounted for 30 percent and sustainability\u2014 based on efficiency in energy, water, CO2<\/sub>, and pesticide usage\u2014accounted for 20 percent.<\/p>\n

A greater journey<\/h3>\n

For Tran and Microsoft, the work demonstrated in the competition is part of a larger commitment to deploying cloud, Internet of Things, and AI technologies to protect and sustain the planet and its natural resources. In July 2017, Microsoft launched AI for Earth (opens in new tab)<\/span><\/a> to support individuals and organizations doing work in water, agriculture, biodiversity, and climate change with grants, education, and further collaboration. The initiative\u2019s strategic approach and funding has since been expanded (opens in new tab)<\/span><\/a>, and the gains being made, especially in the area of data-driven farming, have been impressive. FarmBeats<\/a> is among the projects receiving recognition for its impact, and Tran and another Team Sonoma member, Senior Research Software Engineer Chetan Bansal<\/a>, are also contributors to that work.<\/p>\n

While FarmBeats is improving data collection outdoors with sensors, drones, and other devices for more sustainable farming, Team Sonoma\u2019s work is in controlled environment agriculture (CEA), an enclosed system of farming that allows growers to determine and execute optimal settings for such environmental factors as light, temperature, humidity, and CO2<\/sub> concentration.<\/p>\n

Tran\u2019s interest in CEA as a research area was piqued in 2017, about a year before he had heard about the Autonomous Greenhouse Challenge. As a member of the Reinforcement Learning group with Microsoft Research AI<\/a>, he and his colleagues explore the machine learning technology\u2019s potential for real-world application. Not only is CEA\u2019s ability to have meaningful impact attractive\u2014a more efficient, accessible means to meeting the nutritional demands of the earth\u2019s population\u2014but it is also a great training ground for reinforcement learning models. CEA offers contained scenarios in which to work and an abundance of data, the collection of which is relatively quick and easy thanks to sensing technology and IOT.<\/p>\n

\u201cThe state of the art of reinforcement learning is notoriously data hungry, so it is critical that we focus on new sample-efficient algorithms,\u201d says Tran. \u201c To faster close the gap, though, we also need environments where we can collect a lot of data easily and affordably.\u201d<\/p>\n

\"Tran

Tran (center) with collaborators from Sananbio at a vertical farm inside the company\u2019s Xiamen, China, facility.<\/p><\/div>\n

The application focus of Sonoma<\/a>\u2014the project name for Tran and his colleague\u2019s overall work in the area, as well as the name for the greenhouse challenge team\u2014has been greenhouse and vertical farming (opens in new tab)<\/span><\/a>, both of which have the potential for safer, faster food production with less use of the resources that have been literally the foundation of traditional farming\u2014land and water. According to the greenhouse competition website (opens in new tab)<\/span><\/a>, indoor cultivation such as greenhouse and vertical farming can decrease water requirements by up to 90 percent, needs one-tenth the space to produce the same amount of crop as more traditional farming, and can thrive on less pesticides and chemicals. This promising solution requires a workforce of indoor farming experts that might be outpaced by the demand for indoor farming, though, and Tran has made it Sonoma\u2019s goal to help create autonomy in the space.<\/p>\n

\u201cAI can help both scaling the expert knowledge in developed countries such as the Netherlands to developing countries, but also improve upon the expert growers,\u201d says Tran.<\/p>\n

To reach the Sonoma goal, Tran leads with what he describes as a bottom-up, top-down approach.<\/p>\n

\u201cBy bottom up, we mean doing novel research in reinforcement learning, the very fundamental reinforcement learning research, and also doing the application-centric research simultaneously,\u201d he explains. \u201cApplicable reinforcement learning research is still at a very, very early stage, so there is a lot of ground for new research to be done. For the top-down aspect, our approach is to seek collaboration with domain experts from around the world.\u201d<\/p>\n

The Sonoma project members reflect that philosophy: Tran and Bansal, both of Microsoft Research, represent the AI side. On the plant science side and from partnership institutes are research scientist Xiuming Hao (opens in new tab)<\/span><\/a> of Agriculture and Agri-Food Canada (AAFC) and Chieri Kubota (opens in new tab)<\/span><\/a>, professor of controlled environment agriculture at The Ohio State University, among other collaborators.<\/p>\n

A competition along the way<\/h3>\n

It was Tran\u2019s strong belief in the power of collaboration that led to a Team Sonoma for the international Autonomous Greenhouse Challenge. He was visiting Wageningen University & Research\u2014a leading partnership in food production\u2014in March 2018 to explore opportunities for collaboration. During the meeting, his Wageningen counterparts mentioned the challenge, and he brought it back to his team at Microsoft Research. They were all intrigued.<\/p>\n

\u201cWe started reading about the competition and got excited,\u201d recalls Tran. \u201cIt was a great opportunity to get our feet wet and quickly. There was already a strong commitment from multiple key players in the area\u2014Wageningen University & Research, Tencent, Intel, and Delphy and AgroEnergy, among them\u2014working toward a shared goal and a shared vision. Plus, there was guaranteed support from Wageningen staff during the course of the competition.\u201d<\/p>\n

And Team Sonoma was formed: Tran from the Microsoft Research Redmond lab; Bansal from the Microsoft Research India lab; Thomas Follender Grossfeld (opens in new tab)<\/span><\/a> and Vincent van Wingerden (opens in new tab)<\/span><\/a> of Microsoft Nederland<\/a>; David Katzin, a Wageningen PhD student; and Hong Phan, a PhD student from the University of Copenhagen. Sonoma was one of five teams selected (opens in new tab)<\/span><\/a> from a pool of 15 for the main event after a pre-competition \u201chackathon\u201d that included a virtual growing challenge and a presentation of their approach.<\/p>\n

Each team received 96 square meters of greenhouse space at the Wageningen University & Research campus in Bleiswijk in the Netherlands. Each greenhouse was outfitted with the same system of actuators, including ventilation, heating and artificial lighting, and sensors to measure, among other things, temperature, moisture, and energy consumption. Teams were also permitted to install additional sensors and monitoring equipment.<\/p>\n

Sonoma installed additional cameras but only one additional sensor\u2014a leaf-wetness sensor, which was not among the preinstalled competition sensors. The team chose the sensor for increased monitoring of humidity and moisture, two factors that lead to crop-damaging pests and disease. Teams were permitted inside their greenhouse only once, to set up their extra cameras and sensors. Throughout the competition, from September 1 to mid-December, they ran their AI frameworks remotely.<\/p>\n

But for Bansal, the distance from the site of data collection wasn\u2019t as challenging as the means of data collection itself. The team needed to design a system that could account for Murphy\u2019s law\u2014whatever can go wrong will go wrong.<\/p>\n

\u201cThe data is coming from multiple sources\u2014there are sensor boxes, cameras, the API being used by the greenhouse,\u201d says Bansal. \u201cAll of them can and have failed, so the question is how you\u2019re able to detect it and how fast you\u2019re able to react.\u201d<\/p>\n

\u201cWe had to deal with all of those issues and design a system that was resilient to all of them, but I think that\u2019s part of designing any real-world control system,\u201d Bansal added.<\/p>\n

\"\"<\/p>\n

The Sonoma AI approach<\/h3>\n

Sonoma chose to build its framework around approximate Bayesian model-based reinforcement learning.<\/p>\n

\u201cWe bet on model-based RL because we think it is sample-efficient and generalizable,\u201d says Tran. \u201cSample efficiency is critical for real-world applications. Standard RL algorithms require a huge number of trials\u2014in the millions\u2014to train a good agent, even in simple environments. This is not a big deal in games, where RL has shown success, because an agent can play as many games as it needs. In real-world applications, we cannot afford to run millions of failed trials. So we need to think differently about RL.\u201d<\/p>\n

For reinforcement learning to be a viable solution for today\u2019s societal challenges, the team determined the agent must be initialized as strong as any existing system and have the ability to learn and improve over time without limits to its capacity to reach optimality and conceived a framework (above) incorporating these features.<\/p>\n

The framework begins by training a probabilistic dynamics model. This model learning is analogous to building a simulator, which helps the agent to plan by imagining. In addition, by way of imitation learning, the agent is initialized to perform like an existing expert policy. From there, the agent will operate on a continual model-based policy optimization process, improving its overall performance with every environmental interaction.<\/p>\n

\"From

From left: Research scientist Xiuming Hao, greenhouse vegetable specialist Shalin Khosla and Tran at Agriculture and Agri-Food Canada\u2019s Harrow Research and Development Centre in Ontario<\/p><\/div>\n

For the greenhouse challenge, data from around the greenhouse, such as weather conditions, and from sensors and images from inside the greenhouse were input into the agent, which then determined the intensity and distribution of artificial lighting; the amount of water, CO2, and nutrients to give the plants; and greenhouse temperature. The framework chose settings based on what it learned would result in the most biomass.<\/p>\n

\u201cThe team has successfully incorporated the current best knowledge and best practices on crop cultivation and management and on greenhouse environmental control into its greenhouse AI control system,\u201d says Xiuming Hao, the Agriculture and Agri-Food Canada research scientist who collaborates with the Sonoma project. \u201cThe team started with a high plant density system identified from previous model data, and adjusted the AI climate control based on crop performance and weather conditions over the crop growing period to allow for the best performance of this high density\/high production system.\u201d<\/p>\n

Tran describes the team\u2019s strategy as conservative. The competition setup allowed for only one trial, and there was not much data existing before the challenge; because of that, its strategy relied on a hand-crafted expert policy alone without resorting to reinforcement learning for continuous learning and improvement\u2014yet.<\/p>\n

\u201cBy working with domain experts and leveraging their knowledge, as well as the capabilities of our AI agent, together we were able to produce better results within a short time frame,\u201d says Tran. \u201cAnd this is just the beginning; there is a lot of room for growth, literally and<\/em> figuratively.\u201d<\/p>\n