Beyond the Automated Gaze: Keeping People, Place and Ground Truth in Agricultural Remote Sensing
- Simon Bardsley
- Jul 26
- 5 min read
Updated: Jul 28
A reflection on Terra Analytica: The Automated Gaze of Corn and Soy
I recently came across an academic essay with a title that immediately caught my attention: Terra Analytica: The Automated Gaze of Corn and Soy.
Given that TerraAnalytica is also the name of my Earth observation and geospatial venture in Aotearoa New Zealand, I clearly had to read it.
The essay, written by the artistic research duo FRAUD—Audrey Samson and Francisco Gallardo—does not examine TerraAnalytica the company. Instead, it explores the history and implications of observing agricultural land from above, from early aerial photography and photogrammetry through to satellite imagery, machine learning, crop-yield prediction and geospatial intelligence.
Its central question is an important one:
What happens when systems designed to observe agricultural landscapes begin influencing how those landscapes are valued, managed and changed?
The article was published in Afterimage in 2023 and accompanies FRAUD’s artwork Finis-terra. The work uses woven patterns derived from satellite-based crop-yield predictions to examine how agricultural land can be transformed into classifications, forecasts and financial values.
From interpreting photographs to predicting fields
Long before machine learning, agricultural analysts interpreted aerial photographs using features such as shape, pattern, tone, texture and position.
A skilled photographic interpreter might examine the geometry of a field, differences in vegetation tone, drainage patterns, terrain, crop structure or signs of disease. Different films and filters, including infrared photography, allowed analysts to detect characteristics that were not always visible to the human eye.
The article describes how near-infrared photography could reveal crop stress before symptoms were readily apparent in normal visible imagery. Modern remote sensing scientists will recognise the foundations of today’s multispectral crop-monitoring methods in these earlier practices.
Today, many of these interpretive processes are undertaken by algorithms.
Satellite imagery can be used to:
distinguish crops and land-cover classes;
identify individual field boundaries;
measure crop development through time;
estimate vegetation condition;
forecast production;
assess drought, flooding and other risks;
support agricultural insurance and commodity-market analysis.
This provides extraordinary opportunities. It also creates responsibilities.
A model may appear objective because it produces a number, class or map. But every model contains choices: which data were included, how the training samples were collected, what categories were defined, which uncertainties were accepted and what the result is intended to represent.
The computer has not removed interpretation. It has embedded interpretation within the analytical process.
Agricultural fields contain patterns that can be observed by people, drones and satellites—and increasingly analysed using automated methods.
When observation begins to influence reality
One of the essay’s most compelling ideas is that environmental analytics can become a kind of read–write system.
The satellite reads the landscape. An algorithm turns those observations into classifications or predictions. People then act on those outputs through farming, finance, insurance, investment or policy. Those decisions can alter the landscape that the satellite observes during its next pass.
A crop-yield prediction, for example, may begin as an analytical estimate. Once it influences an insurance premium, lending decision, commodity position or land-management response, it becomes part of the system it was intended to describe.
This does not make crop modelling inherently harmful or unreliable. It means that we must clearly distinguish between:
what the sensor observed;
what the model inferred;
how uncertain that inference is;
and what somebody subsequently decides to do with it.
Those four stages are related, but they are not the same thing.
Observing a maize field from the ground provides context that cannot be captured by satellite imagery alone. Hauraki Plains, New Zealand.
What a satellite cannot see on its own
A satellite image may show that one part of a maize field reflects differently from another. It cannot independently explain why.
The variation might relate to:
planting date;
soil type;
waterlogging;
drought stress;
nutrient availability;
disease;
canopy structure;
crop development;
row orientation;
recent weather;
harvest activity;
sensor geometry;
shadows or atmospheric effects.
A machine-learning model may identify a statistically useful relationship, but determining the physical or agricultural cause requires additional evidence.
This is where ground truth remains essential.
For TerraAnalytica’s MaizyHaz research on the Hauraki Plains, satellite observations are being considered alongside field photographs, environmental observations, weather records, drone imagery, plant measurements and available harvest information.
The aim is not simply to produce a map or classification. It is to understand whether patterns identified through Earth observation have a defensible relationship with conditions observed in the field.
A model may successfully separate different seasonal or spatial patterns within a dataset, but that does not automatically explain what those differences represent. It could be responding to planting time, soil moisture, crop management, field location, environmental stress or image timing.
Responsible analysis means testing competing explanations rather than accepting the most convenient one.
Ground observations help determine whether variation identified in satellite imagery reflects crop condition, soil moisture, management or another environmental factor.
The importance of standing in the field
Remote sensing offers a remarkable perspective. It allows us to see spatial relationships, regional patterns and seasonal change that may be impossible to recognise from the ground.
But the view from above becomes much more meaningful when it is connected with the view from within the landscape.
Walking through a maize field reveals things that are compressed or invisible in a satellite pixel: the height and density of the plants, leaf condition, standing water, soil texture, damaged areas, weeds, gaps between rows and differences in crop maturity.
It also reconnects the analysis with place.
The Hauraki Plains are not simply a collection of raster cells or agricultural polygons. They are a lived landscape shaped by rivers, drainage, soils, weather, farming, communities and generations of environmental change.
Good Earth observation should not erase that complexity. It should help us understand it.
Towards responsible agricultural Earth observation
Reading The Automated Gaze of Corn and Soy reinforced several principles that I believe should guide TerraAnalytica’s work.
Ground observations should remain part of the evidence
Satellite data are powerful, but field observations help determine what remotely sensed patterns actually represent.
Predictions should include uncertainty
A classification probability or yield estimate should not be presented as certain simply because it came from an algorithm.
Model limitations should be visible
Users should be able to understand the major assumptions, training-data limitations and potential sources of error.
Local knowledge matters
Farmers, landholders and people familiar with the landscape may recognise causes and historical conditions that are absent from the dataset.
Measurements and decisions should remain distinct
A vegetation index is a measurement derived from reflected energy. It is not, by itself, a judgement about crop value, farm performance or land-management quality.
Models should be explainable where possible
Techniques such as variable importance, permutation testing and SHAP analysis can help show which environmental and remotely sensed variables are influencing a classification.
Data should be used with purpose and care
The ability to observe or classify a field does not automatically determine how the information should be stored, shared, commercialised or used.
More than an automated gaze
The phrase “automated gaze” is deliberately challenging. It asks whether geospatial technology distances us from the landscapes and lives being analysed.
I do not believe that distance is inevitable.
Remote sensing can support environmental care, agricultural research, better resource management and a deeper understanding of Earth systems. It can reveal crop stress, flood impacts and seasonal change. It can help direct fieldwork and make limited resources more effective.
But this requires us to treat remote sensing as a relationship between measurements, models, landscapes and people—not merely as a stream of pixels waiting to be monetised.
For TerraAnalytica, the goal is not simply to look at the land from above.
It is to connect that view with evidence collected on the ground, explain what we know and do not know, and produce analysis that remains accountable to the places it represents.
That is the difference between an automated gaze and a genuinely grounded form of Earth observation.
FRAUD Article
Post written with help from Lila (ChatGPT), Chief of Communication.
Comments