The Planet We Reconstructed After It Happened

AN INCONSISTENT PAST

The observational record this book has spent two volumes assembling has a defect that makes it treacherous for studying change. The instruments kept improving. Stations opened and closed. Satellites launched, aged and were replaced. A trend computed directly from raw observations partly measures the climate and partly measures the history of the observing system.

Reanalysis is the response, and it is one of the odder ideas in Earth science: take every historical observation, feed it into a single modern forecast model held fixed, and let the model produce a physically consistent estimate of what the whole planet was doing at every hour, including everywhere nobody was watching.

DATA ASSIMILATION

How observations become a state

The machinery underneath is data assimilation, which is also how every operational forecast begins and deserves stating plainly because it is widely assumed to be simpler than it is. Observations do not become a forecast's starting state directly. They are combined with a short forecast from the previous cycle, weighted by the estimated error of each, to produce a best estimate of the current state that is consistent with both the physics and the measurements.

This is necessary because observations are scattered, irregular, and vastly outnumbered by the grid points a model needs filled. There are no measurements over most of the ocean surface at most hours. The model supplies the rest, constrained by physics, and the observations pull it toward reality wherever they exist.

It also means every gridded product of this kind is a hybrid: partly measurement, partly the model's opinion, in proportions that vary by location, altitude and era, and which the product does not label. A user reading a value over the mid-Pacific in 1955 is mostly reading physics with a little data attached.

THE MODERN RECORD

What ERA5 actually is

ERA5, produced by ECMWF, is the reanalysis most of this field now runs on. It provides hourly values of surface and upper-air variables at roughly 31-kilometer global resolution across 137 vertical levels, extending back to 1940, assimilating all available observations both in situ and from satellites, with the assimilation combining a previous forecast with new observations at regular cycles to produce each successive best estimate.

The consistency is the product. Because one fixed model version processes the whole period, a trend computed from ERA5 is not contaminated by the model changing underneath it, even though the observations feeding it changed enormously between 1940 and now. What remains is that the estimate is better constrained in recent decades, where satellites saturate the globe, than in the 1950s, where it is largely physics filling in around sparse measurements.

This matters far beyond climate research, because ERA5 is the training data for the machine learning weather models that arrive in Volume 5. Those models did not learn from the atmosphere. They learned from a reconstruction of the atmosphere, which inherits every assumption in the model that produced it.

WHAT KIND OF OBJECT THIS IS

A history that is physically consistent and partly invented

Reanalysis produces something with no clean precedent: a complete, gap-free, physically coherent account of a past that was never observed completely. It is not a measurement and not a simulation. It is the most probable past given the physics and whatever was recorded, which is a legitimate and useful thing to compute, and a strange thing to treat as ground truth.

The field mostly does treat it as ground truth, because the alternative is worse and because a consistent estimate is enormously more usable than an honest scatter of gaps. The cost is that a subtle error in the model becomes a subtle error in the historical record, and then becomes a subtle error in every model trained on that record, with no observation anywhere in the chain to contradict it.