The Model Learned Time

TEMPORAL LEARNING

A single observation can tell a model what a place looks like. A sequence can tell it whether the place is growing, drying, flooding, recovering, being built, being harvested or simply passing through a normal season. Time turns appearance into behaviour.

That distinction matters because many Earth classes are ambiguous in one frame. Bare soil can be a construction site, a harvested field or a dry lake bed. Water can be permanent, seasonal or an event. Green vegetation can be a crop, pasture or regrowth. The answer often lives in what happened before and after.

The first machine-learning systems mostly treated repeated observations as extra inputs. The deeper shift was to treat order itself as information. Once the model knows that one state followed another, it can learn phases, persistence, transitions and anomalies that no still image contains.

HOW MODELS CARRY TIME

From summary statistics to temporal state

An early way to use a year of imagery was to collapse it into statistics: maximum greenness, minimum moisture, seasonal amplitude, median reflectance. This is efficient and sometimes excellent. It is also destructive. Two sequences with the same maximum and minimum can represent entirely different events if one changed gradually and the other changed overnight.

Recurrent neural networks addressed that by carrying a hidden state from one observation to the next. Temporal convolutional networks look for patterns across neighbouring dates. Attention-based models compare observations directly and learn which dates matter most. The mechanism differs, but the objective is similar: preserve enough order that the model can recognize a trajectory rather than a pile of measurements.

The Temporal Convolutional Neural Network proposed by Pelletier and colleagues for satellite image time-series classification showed why this matters for remote sensing. Instead of relying on a hand-designed seasonal summary, the network learned discriminative temporal patterns directly from the sequence. The model could use when a signal changed, how quickly it changed and how long it remained in a state.

The later Pixel-Set Encoder with Temporal Attention Encoder made the selection more explicit. Its attention mechanism can give more weight to informative dates and less to ordinary ones. A crop field does not need every day of the season to explain itself equally. Planting, rapid growth, peak canopy and harvest carry more information than the weeks in between. The model learns that calendar without being handed a crop calendar.

IRREGULAR TIME

Earth does not provide evenly spaced frames

The diagrams usually show a neat sequence. Earth rarely provides one. Optical scenes disappear behind cloud. Different satellites pass at different intervals and local times. Stations fail. A field survey may happen twice in a season while a soil probe reports every fifteen minutes. Even the same sensor may have a long gap at exactly the moment something important happened.

A temporal model therefore has to distinguish absence of evidence from evidence of stability. If there was no usable image for three weeks, carrying the last observation forward silently assumes nothing changed. Interpolating between observations assumes change was smooth. Both choices create state the instrument never measured.

Modern sequence models can encode the observation time, the gap since the previous observation and masks describing what is missing. Attention can work directly on the observations that exist instead of forcing everything onto a perfectly regular calendar first. This does not solve missing data. It stops the preprocessing pipeline from pretending missing data never happened.

Missingness can itself contain information. Monsoon cloud is seasonal. Smoke can obscure a landscape during a fire. Persistent cloud can concentrate over particular terrain. A model that treats every missing scene as random may erase part of the process it is trying to learn. The blank spaces have a history too.

CHANGE AND PHASE

Time separates a condition from a transition

Temporal intelligence changes the question from what is here to what is happening here. That sounds like a small grammatical improvement and is actually a different class of problem. A reservoir that is low but rising means something different from one at the same level and still falling. A construction site that has been bare for six months means something different from one cleared yesterday.

Change detection traditionally compares two dates. Sequence models can estimate something richer: when the change began, whether it persisted, whether it reversed and whether it fits the location's normal seasonal cycle. A temporary flood, a permanent new water body and a cloud-shadow artefact can all look like dark pixels in a pair of images. Their trajectories are very different.

This is where temporal models become useful for monitoring rather than mapping. A map assigns state. A monitoring system maintains state and updates it as evidence arrives. The difference is operational. A static map can be excellent and still be obsolete. A temporal model is expected to notice when its own previous answer has expired.

That expectation introduces a harder problem: concept drift. The patterns used to identify a phase can themselves change. Planting calendars shift. Reservoir operations change. Cities expand. Climate alters seasonal timing. A model trained to recognize yesterday's normal can eventually classify tomorrow's normal as an anomaly with great confidence.

THE BOUNDARY

Memory is useful until history changes its rules

Temporal models learn regularity from the historical sequence. That gives them an enormous advantage over static recognition, but it also ties them to the statistical world that produced the sequence. If a relationship changes, memory can become a source of error rather than protection from it.

The problem is not unique to Earth observation. Any learned time-series model can mistake persistence for law. Earth makes the problem unusually awkward because the system is being altered while the model watches: climate, land use, infrastructure, policy and human behaviour all shift the baseline.

A useful planetary system therefore needs two kinds of memory. It needs enough history to know what usually happens, and enough skepticism to notice when the usual relationship has stopped holding. The second ability cannot come from time alone. It requires other kinds of evidence.

That is the next step in the architecture. The same place is observed not only across dates, but through different physical channels. Radar, optical imagery, weather, elevation and field measurements do not merely extend the sequence. They disagree in informative ways. The model now has to learn how several languages describe one event.