Tandon researcher joins major collaboration aimed at using AI models to improve agriculture

Chinmay Hegde, professor of computer science and engineering and electrical and computer engineering at the NYU Tandon School of Engineering is part of a multi-institutional collaboration to pursue foundational advances in artificial intelligence (AI) to enhance the resiliency of the nation’s agricultural ecosystem.

The program, AIIRA: AI Institute for Resilient Agriculture, is a national Research Institute supported by a $20M, five-year grant from the U.S. National Science Foundation. It is led by Iowa State University, and includes researchers at Carnegie Mellon, the University of Arizona, George Mason University, and the University of Nebraska, along with several industry and federal partners.

Driving these advances is the concept of biophysics-aware AI-driven digital twins that are capable of assimilating sensor data along with agronomic knowledge to build accurate models of agricultural phenomena across scales, from individual plants to plots/fields. 

Hegde explains that digital twins, AI-driven simulations of the agricultural lifecycle — from seed to plant to field to harvesting to supply chain and distribution — constitute potentially vital tools for improving outcomes and costs, but that creating them requires the innovative use of training data in AI.

Hegde explained that the key technical novelty his team will exploit involves biophysics-aware AI modeling that will augment limited training data with domain expertise.

“Most use cases for training AI typically involve images or text fragments with billions of data points, but for applications such as agriculture, we don't have, for example, access to billions of images of a plant or field; with these ‘low data’ challenges, the idea is to make up for what we don’t have with domain knowledge: physics models for plants, such as rate of growth, genomics, and responses to stresses.”

He added that the project will also collect image data from cameras deployed in fields and at other touch-points in the agricultural farm-to-home production and delivery chain. The end goal, he explained, is optimal decision-making at each stage of the agricultural process.

“Even routine decisions such as whether to plant or pick a particular harvest right now, or wait for a week — decisions that cumulatively make a large economic impact — are often driven by intuition,” he said. “And they are often made by people without training or domain knowledge in things like optimal harvesting schedules for a given location under the prevailing stress factors. (AI-driven models) could make holistic improvements to the system leading to faster and better decisions not limited to human intuition.”

Hegde will be leading two main research thrusts:

• The first thrust will involve building novel AI theory and algorithms that code contextual knowledge into AI models. These will lead to new deep, biophysics-aware generative models, building upon their preliminary work on learning fast AI surrogates for engineering systems described by partial differential equations (PDEs).
• The second thrust will involve developing foundational algorithms and theory to increase trustworthiness of AI-enabled technologies among the agricultural community. This will build upon their preliminary work on developing AI-based computer vision systems that are resilient to input perturbations (such as changing illumination, weather conditions, or other visual degradations.)