Helmets Labeling Crops: A New Frontier in Agricultural Monitoring

Faculty specialist Diana Botchway Frimpong spotlights new paper on helmet-mounted cameras and deep learning for scalable crop mapping.

In the face of escalating climate extremes such as droughts and floods, accurate and timely agricultural monitoring has never been more critical. In Africa, where smallholder farmers form the backbone of the food system, reliable information on crop distribution is key to planning, resilience and food security. Yet the creation of crop-type maps has long been limited by the high cost and time requirements of traditional field surveys.

Our recent paper, “Helmets Labeling Crops,” details our approach to collecting crop-type datasets via helmet-mounted cameras and deep learning. This method offers a low-cost, scalable approach for generating open, validated crop-type datasets.

 Collecting agronomic data using traditional methods.

Led by Assistant Professor Catherine Nakalembe, the research team included faculty specialist Ivan Zvonkov, Assistant Professor Hannah Kerner (Arizona State University), me [faculty specialist Diana Botchway Frimpong], Kenneth Mwangi (World Resources Institute), Jane Kioko ( Ministry of Agriculture and Livestock Development, Nairobi, Kenya), Bhanu Tokas, Kartik Jawanjal, Anjali Paliyam (B.S. Computer Science, Spring ‘25), Christopher Atsianzale Wakhanala (B.S. Geographical Sciences, Spring ‘25), Iman Smith (M.S. Geographical Sciences, Spring ‘25), Assistant Research Professor Ana Tarano (Arizona State University), Shreya Jha (B.S Geographical Sciences, Spring ‘25), and Priscilla Loh (PhD Student, Southern University and A &M College), who together coordinated fieldwork, image processing and data validation across multiple counties.'

This diagram summarizes our approach; collecting geotagged street-level images using GoPro cameras mounted on motorcycles or other moving vehicles.

A Practical, Low-Cost Data Approach

The innovation at the heart of this work is simple. Field agents wore helmet-mounted GoPro cameras while riding along rural roads during the 2021 and 2022 long-rain seasons. These cameras captured geotagged images of nearby fields, which were then processed using a deep learning pipeline adapted from the Street2Sat framework.

In total, we collected 32,804 GoPro images from focus counties in Kenya and processed them through our automated pipeline. Of these, 28,580 images were filtered out as unsuitable for conversion into crop-type points. Our Street2Sat Label Squad which includes undergraduate, graduate and early-career researchers, contributed to reviewing and improving the data sets.

"Working as an undergraduate research assistant with the Street2Sat project was both eye-opening and rewarding,” said Anjali Paliyam, GEOG undergraduate student. “I helped with satellite image interpretation and Python-based improvements to our machine learning pipeline, including a duplicate point resolution method that made verification more efficient.”

Extension agents on the field collecting crop type data using GoPro cameras in Tanzania, Uganda, Kenya and Nigeria.

From Field to Reliable Data

Ensuring reliability was a core priority. Each georeferenced point was reviewed in Google Earth Pro, where the captured field imagery was compared against satellite data. A two-reviewer system, with additional oversight from a partner at Kenya’s Ministry of Agriculture, ensured misclassified or off-target images were corrected or excluded.

This process allowed us to build confidence in the dataset, which includes a rich mix of crops: 4,351 maize points, 301 sugarcane, 140 banana, 106 wheat and smaller numbers for crops such as tea, beans, sunflower, rice, soybean, cassava and kale. This careful validation not only strengthened the dataset but also gave team members hands-on experience with innovative field and data collection methods.

“As a recent graduate, contributing to this project was an exciting opportunity for me to explore a different way of collecting and processing agricultural data,” said Iman Smith (M.S. Geographical Sciences ‘25). “Working with helmet-mounted cameras and deep learning showed me how creative field methods can generate high-quality datasets that support food security research.”

Team of reviewers who perform daily data quality assessments on images captured and processed. From left: Anjali Paliyam, Diana Botchway Frimpong, Ivan Zvonkov and Christopher Atsianzale Wakhanala.  Zoom Image from left: Priscilla Loh, Iman Smith, Shreya Jha and Dr. Catherine Nakalembe. 

Demonstrated Utility: Mapping Maize

To demonstrate practical use, we trained a random forest classifier to map maize in Nakuru County. Despite challenges posed by heterogeneous, smallholder-dominated fields, the model achieved an overall accuracy of 71.1%, with producer accuracy of 58.2% and user accuracy of 45.7%. These results underscore both the promise of the dataset and the inherent challenges of mapping crops in complex landscapes.

Open Science and Global Expansion

This approach is now being scaled beyond Kenya. Similar campaigns are underway in Uganda, Tanzania, Zambia, Nigeria, Germany, Senegal, India, South Korea and Bhutan. This demonstrates the adaptability of the helmet-camera approach to diverse geographies and farming systems.

Why This Work Matters

1. Cost-effectiveness: Lowers the financial barrier for agricultural data collection.
2. Scalability: Deployable across regions using local field agents and citizen scientists.
3. High-quality, open data:  Rigorously validated and openly shared for global benefit.
4. Bridging field and space: Provides the ground truth needed to unlock satellite monitoring potential.
5. Supporting food security: Offers a foundation for better crop forecasting and climate resilience.

By reimagining data collection with simple tools like helmet-mounted cameras, our team has shown how innovation and collaboration can democratize agricultural monitoring. The Helmets Labeling Crops dataset is more than a research output; it is a blueprint for making agricultural data collection affordable, scalable and impactful worldwide.

Read the Full Paper

"Helmets Labeling Crops: Kenya Crop Type Dataset Created via Helmet-Mounted Cameras and Deep Learning." Nakalembe, C., Zvonkov, I., Kerner, H. et al. Sci Data 12, 1496 (2025). https://doi.org/10.1038/s41597-025-05762-7 

Main image of a extension agent on the field collecting crop type data using a GoPro camera. All images courtesy of Catherine Nakalembe and Diana Botchway Frimpong.
 

Published on Fri, 09/05/2025 - 13:44